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1.  INTRODUCTION 


Prostate  cancer  is  the  second  leading  cause  of  cancer  death  and  the  most  common  cancer  detected  in  men  in  the 
United  States.  An  estimated  238,590  new  cases  of  prostate  cancer  will  be  diagnosed  in  the  United  States,  and 
approximately  29,720  men  will  die  of  prostate  cancer  during  2013.  Radical  Prostatectomy  (RP)  aims  for 
complete  cancer  resection  and  has  been  shown  to  improve  cancer  survival.  Robotic-assisted  laparoscopic 
prostatectomy  (RALP)  has  recently  emerged  as  an  alternative  to  open  and  laparoscopic  procedures.  The  daVinci 
Surgical  System  (Intuitive  Surgical,  Sunnyvale,  CA)  provides  3-D  visualization,  higher  magnification,  hand 
tremor  elimination  and  refined  dexterity  by  incorporating  wristed  instrumentation.  From  250  robotic  cases  in 
the  beginning  (2001),  the  number  has  reached  73,000  in  2009  (86  %  of  the  85,000  American  men  who  had 
prostate  cancer  surgery).  Alternative  surgical  systems  are  in  the  work  and  are  awaiting  FDA  approval. 

Initial  experiences  with  the  daVinci  surgical  system  have  been  positive:  short  learning  curve,  limited  blood 
loss,  less  post-operative  pain,  favorable  complication  rates,  and  short  hospital  stay.  Despite  fewer  perioperative 
complications  and  shorter  hospital  stay,  a  recent  paper  found  patients  were  three  times  more  likely  to  require 
salvage  therapy.  One  theoretical  disadvantage  with  regards  to  robotic  surgery  is  the  lack  of  tactile  feedback.  In 
open  RPs,  the  surgeon  uses  his  fingers  to  feel  the  periphery  of  the  prostate  gland.  Without  tactile  feedback,  a 
robotic  surgeon  is  unable  to  appreciate  differences  in  tissue  texture  or  firmness  and  therefore  may  not  be  able  to 
tailor  precisely  the  extent  of  tissue  excision  around  the  prostate  gland  in  efforts  to  eradicate  all  cancerous  tissue. 
Inadvertently  leaving  residual  cancer  cells  behind,  called  a  positive  surgical  margin  (PSM),  is  highly  associated 
with  cancer  recurrence.  PSM  rates  were  initially  higher  in  RALP  than  in  the  open  procedure,  but  they  have  been 
shown  to  decrease  with  surgeon's  experience  and  improved  technique. 

As  manual  palpation  helps  guide  the  surgeon  in  the  open  procedure,  an  equivalent  real-time  guiding  tool  is 
needed  for  robotic  prostatectomy.  Imaging  modalities  like  MRI  or  CT  are  not  feasible  intra-operatively,  nor  do 
they  possess  the  sensitivity  or  specificity  for  accurate  detection  and  localization  of  prostate  cancer.  Trans-rectal 
ultrasound  (TRUS)  is  routinely  used  in  diagnosis,  in  conjunction  with  digital  rectal  examination  (DRE)  and 
biopsies.  One  center  used  TRUS  for  real-time  monitoring  and  guidance  during  Laparoscopic  RP  and  reported 
technical  feasibility  and  enhanced  precision  by  decreased  PSM  rates.  TRUS  was  capable  of  imaging  a 
substantial  percent  of  non-palpable  prostate  cancers.  The  authors  recognized  however,  the  limitations  of  TRUS 
guidance;  it  requires  considerable  prior  expertise  and  tends  to  identify  primarily  hypoechoic  lesions,  which  were 
just  47\%  of  the  cancer  nodules  studied.  Today's  prostate  cancer  patients  are  more  likely  to  present  with 
echogenic  or  isoechoic  lesions  because  aggressive  screening  techniques  lead  to  a  shift  toward  smaller,  early- 
stage  cancers  ;  classic  B-mode  gray-scale  ultrasound  alone  cannot  identify  these  lesions. 

Ultrasound  (US)  Elastography  (USE)  is  emerging  as  a  valuable  tool  in  the  field  of  imaging.  Elastography  is  a 
qualitative  technique  based  on  the  principle  that  tissue  compression  produces  strain  (displacement)  within  that 
tissue;  strain  is  smaller  in  harder,  stiffer  tissue  than  in  softer,  more  compliant  tissue.  Analyzing  the  ultrasound 
raw  radio  frequency  signal  results  in  a  strain  map,  commonly  called  elastogram,  where  harder  tissue  is  darker 
than  surrounding  soft  tissue.  Cancers  tend  to  present  as  hard  lesions  due  to  increased  cellularity.  Echogenicity 
and  stiffness  of  tissue  are  generally  uncorrelated;  USE  can  identify  hypoechoic  lesions,  but  also  echogenic  or 
isoechoic  cancers  that  classic  gray-scale  ultrasonography  cannot.  Elastography  through  the  trans-rectal  approach 
has  already  been  proven  feasible  in  guiding  biopsies  of  the  prostate.  Integrating  USE  technology  with  a 
laparoscopic  ultrasound  probe  will  give  robotic  and  laparoscopic  surgeons  an  important  image-guidance  tool, 
which  until  this  point  does  not  exist. 

There  is  a  need  for  real-time  intraoperative  guidance  in  laparoscopic  prostatectomies.  Hence  the  objective  of 
this  proposal  was  to  develop  such  a  capability.  USE  technology  can  be  integrated  with  the  robotic  probe. 
Elastography  is  ideal  as  a  technology  as  it  allows  for  real-time  acquisition  of  images  of  the  prostate  gland  and, 
similar  to  human  palpation,  allows  for  contact  based  interrogation  of  the  prostate’s  surface.  USE  is  affordable 
and  minimally  invasive  and  can  also  be  miniaturized  and  incorporated  into  robotically  assisted  prostatectomy. 
USE  using  a  laparoscopic  ultrasound  probe  (LAPUS)  can  help  the  surgeon  visualize  the  anatomy  of  the  prostate 
gland,  identify  the  contours  of  the  cancerous  tumors  as  well  as  any  extra  capsular  extension.  The  contribution  of 


the  proposed  research  project  is  in  allowing  for  real-time  intraoperative  acquisition  of  images  of  the  prostate 
gland  and  the  surrounding  tissues.  We  are  proposing  a  safe,  simple  and  robust  technique  for  direct  interrogation 
of  the  prostate  surface  aided  by  a  laparoscopic  US  probe.  USE  can  be  a  valuable  tool  in  the  identification  of 
cancerous  lesions  and  the  trajectory  of  cavernous  nerves  and  thus  improve  the  chances  for  a  cancer- free,  nerve¬ 
sparing  outcome. 


2.  BODY 

This  final  report  will  address  each  of  the  tasks  we  proposed  to  accomplish  in  the  Statement  of  Work  as 
outlined  in  Table  1.  Additional  research  accomplishments  will  also  be  reported. 


Year  1 

Year  2 

Year  3  (9  mouths) 

Task  2.  System  Integration  and  Clinical  Evaluation 

2a.  Integration,  testing,  pre-operative  plan 

2b.  IRB  approval 

2c.  Clinical  smdy  (N=  5-10) 

Task  3.  Design  and  build  new  LAPUS  probe 

3a.  Evaluate  data  from  clinical  study 

3b.  Design  and  build  New  LAPUS  probe 

3c.  Automatic  robotic-assisted  palpation 

Task  4.  Laparoscopic  Prostatectomy  with  Advanced  US  Imaging  Me 

ihods 

4a.  Probe  evaluation  -  phantoms 

4b.  Probe  evaluation  -  animal  models 

4c.  Cavernous  nerves  imaging 

Table  1 :  Statement  of  Work  for  Engineering  Tasks  2,3,4 


Task  1.  Education  and  Training  (Year  1-3) 

Throughout  the  two  years  and  nine  months  of  the  award,  I  have  been  constantly  learning  and  improving 
as  a  prostate  cancer  scientist.  I  had  regular  meetings  with  my  mentors,  Dr.  Allaf,  Dr.  Macura  and  Dr.  Boctor 
which  provided  me  with  urology  and  radiology  insights.  I  completed  relevant  coursework  {Surgical  Robotics, 
Computer  Integrated  Surgery  Seminar)  and  also  the  education  needed  to  prepare  and  propose  to  the  IRB  for 
human  and  animal  studies  (Human  subject  research,  Animal  safety,  Radiation  safety,  Animal  care  and  use, 
Research  ethics,  Privacy  issues,  Blood  borne  pathogens).  The  biggest  gains  were  made  however  discussing  the 
direction  and  means  of  accomplishing  our  study's  goals.  I  spent  valuable  time  with  the  Johns  Hopkins  Hospital 
Pathology  department,  where  I  have  witnessed  firsthand  both  the  macro  pathology  and  the  micro  pathology 
processing  of  prostate  specimens.  The  pathology  staff  and  doctors  provided  me  with  step  by  step  directions 
through  the  process  which  was  very  helpful  for  developing  a  study  in  which  ultrasound  imaging  would  not 
hinder  subsequent  pathological  processing.  In  preparing  for  the  animal  study  I  also  interacted  with  surgeons 
skilled  in  radical  prostatectomy.  These  discussions,  coupled  with  operating  room  observations  with  Dr.  Allaf 
and  Dr.  Su  are  the  foundation  of  my  prostate  cancer  education  and  training.  With  respect  to  my  engineering 
education  and  training,  I  have  completed  all  the  courses  recommended  by  my  mentors  but  I  continue  to  learn  by 
participating  in  departmental  and  surgical  robotics  specific  seminars,  in  which  I  am  enrolled  every  semester. 

In  the  last  three  years  I  have  attended  the  following  conferences: 

1.  PCRP  Innovative  Minds  in  Prostate  Cancer  (IMPaCT)  conference  Orlando,  FL  in  201 1. 1  presented  (oral 
presentation  and  poster)  the  proposal  for  this  award.  Attached  in  the  appendix  is  the  poster  presentation. 

2.  In  2012, 1  attended  the  SPIE  Medical  Imaging  Conference  which  provided  me  with  insight  into  the 
frequent  collaborations  between  engineers  and  physicians. 


Task  2.  System  Integration  and  Clinical  Evaluation  (months  1-12) 

We  proposed  an  integration  of  an  existing  laparoscopic  ultrasound  probe  (Intuitive  Surgical,  Sunnyvale, 
CA)  of  7.5  MHz  frequency  with  the  daVinci  surgical  system.  Task  2a:  Integration,  testing  (months  1-6)  has 
been  done  successfully  in  our  lab  in  201 1,  as  shown  in  the  publication  submission  in  Appendices. 

Billings,  S.,  Nishikant,  D.,  Kang,  H.  J.,  Taylor,  R.,  Boctor,  E.,  " System  for  robot-assisted  real-time 
laparoscopic  ultrasound  elastography,"  presented  and  published  at  SPIE  Medical  Imaging  Conference,  2012. 

Figure  1  shows  a  block  diagram  of  our  complete  system  for  robot-assisted  ultrasound  elastography.  The 
daVinci  S  surgical  robot  is  the  primary  component  of  our  system.  The  experimental  “Read/Write”  Research 
Application  Programming  Interface  (API)  provided  by  Intuitive  Surgical  Inc.  was  used  to  enable  robot  motion 
to  be  controlled  from  computer  in  addition  to  control  inputs  from  the  master  console.  The  daVinci  robot 
manipulates  a  prototype  version  of  a  robotically  articulated  laparoscopic  ultrasound  probe,  also  developed  by 
Intuitive  Surgical  Inc.,  which  was  built  into  the  form  factor  of  a  standard  daVinci  tool.  The  ultrasound  probe  is 
driven  by  a  Sonix  RP  ultrasound  system  (Ultrasonix  Medical  Corp.,  Richmond  BC  Canada),  which  provides  an 
Ultrasound  Research  Interface  granting  access  to  pre-beam  formed  RF  data  from  the  ultrasound  probe.  The  RF 
is  read  and  processed  in  real  time  using  elastography  algorithms  developed  in  our  lab.  A  high-performance 
external  NVIDIA  Tesla  series  GPU,  which  connects  locally  to  the  Sonix  RP  system,  provides  sufficient  parallel 
computing  power  to  generate  strain  images  from  RF  data  in  real.  The  elastography  images  from  the  GPU  are 
streamed  from  the  Sonix  RP  system  to  a  network  port  on  the  system’s  central  workstation.  The  workstation  also 
receives  a  parallel  stream  of  conventional  B-mode  ultrasound  images  from  the  Sonix  RP  machine  directly.  The 
workstation  sends  both  image  streams  to  picture-in-picture  overlays  shown  in  the  stereo  display  of  the  daVinci 
console  (Figure  2).  The  picture  overlays  enable  the  surgeon  to  observe  the  ultrasound  and  elastography  image 
feeds  in  real-time  without  releasing  control  of  the  robot  arms  or  diverting  attention  away  from  the  task  at  hand. 
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Figure  1 .  Block  diagram  of  the  robot-assisted  system  for  real-time  ultrasound  elastography 


Figure  2.  View  of  the  daVinci  console  display  during  a  test  with  an  elasticity  phantom;  the  elastography  image 
overlay  differentiates  lesions  of  different  stiffness. 

Part  of  task  2a  was  to  work  with  Dr.  Allaf  and  our  clinical  team  to  put  together  a  plan  to  be  carried  into 
the  operating  room  for  testing  of  the  laparoscopic  ultrasound  probe  in  a  clinical  setting.  This  plan  would  have 
been  done  in  conjunction  with  task  2b:  IRB  approval  (months  1-6)  in  which  Institutional  Review  Board 
approval  for  a  small  (N=5-10)  clinical  study  using  the  LAPUS  probe  was  going  to  be  sought  and  the  subsequent 
2c:  Clinical  evaluation  (months  6-12).  Given  the  successful  system  integration,  the  implementation  of 
automatic  palpation  and  the  successful  testing  on  tissue  mimicking  phantoms,  our  team  felt  the  next  step  would 
be  to  test  the  system  on  ex-vivo  prostate  specimens.  This  comes  as  an  intermediary  step  and  a  natural  bridge  to 
clinical  testing.  At  the  end  of  the  first  year  of  the  award,  we  had  not  yet  achieved  tasks  2b  and  c.  We 
experienced  delays  in  the  submission  of  the  IRB  due  to  institutional  issues.  As  the  Johns  Hopkins  Hospital 
(JHH)  performs  hundreds  of  prostatectomies  every  year,  we  were  pressed  to  come  up  with  a  process  which 
would  not  disrupt  the  workflow  of  the  Pathology  department  and  would  not  extend  the  surgical  procedures  more 
than  10-15  minutes.  Our  urology  team  lead  by  Dr  Allaf  was  confident  they  could  perform  the  prostate  gland 
interrogation  with  minimum  impact  in  the  duration  of  the  surgery.  The  Pathology  department  was  concerned 
with  the  additional  processing  of  the  specimens.  We  responded  by  designing  a  phantom  box  to  improve  the 
process  so  that  pathologists  could  process  the  specimens  faster  and  with  higher  accuracy.  For  our  study  only,  a 
trained  resident  pathologist  would  have  had  to  create  a  3D  map  of  the  pathological  findings,  which  was  needed 
in  order  to  align  and  compare  the  Elastography  findings  with  the  current  gold  standard  represented  by  the 
pathology  report.  This  is  not  currently  the  standard  procedure  at  JHH  so  this  was  viewed  as  a  hurdle  for 
approval  of  the  study  by  the  JHH  Review  Board. 

Having  been  experiencing  delays  in  the  submission  of  the  IRB  proposal,  out  team  debated  in  year  two  of  the 
award  how  to  best  accomplish  the  proposed  project  goals  in  the  time  left  available.  Our  goals  were  to  evaluate 
the  capabilities  of  the  Laparoscopic  Ultrasound  (LAPUS)  probe  (Figure  3)  in  imaging  prostate  cancerous  tissue 
and  also  to  prepare  for  clinical  testing  of  LAPUS  as  an  imaging  tool  during  robotic-assisted  laparoscopic 
prostatectomy. 


Figure  3:  LAPUS  Probe 

In  our  clinical  study,  the  standard  ultrasound  probe  Siemens  VF10-5  linear  array  was  used  as  a  surrogate 
transducer,  with  similar  capabilities  with  the  LAPUS  probe.  Existing  ultrasound  data  from  ex-vivo  prostate 
specimens  (collected  recently  by  our  team  as  part  of  an  ongoing  study)  were  used  to  assess  the  capability  of  the 
standard  VF10-5  probe.  N  =  6  prostate  specimens  from  patients  enrolled  in  our  study,  provided  N  =  10  target 
areas,  8  hard  nodules  in  the  peripheral  zone,  1  hard  and  1  soft  nodule  in  the  central  part  of  the  gland.  The  results 
of  the  study  will  be  presented  below,  at  task  3a. 

Task  3.  Design  and  build  new  LAPUS  probe  (months  13-24) 

Data  from  the  ex-vivo  prostate  specimens  study  was  evaluated  (Task  3a)  and  the  elastography  findings 
compared  favorably  with  pathology  reports  and  MRI  findings  -  Figure  8  and  Table  2.  Our  analysis  proved  that 
an  ultrasound  probe  fitted  with  a  transducer  similar  with  our  LAPUS  probe  can  produce  elastograms  which  can 
help  the  surgeon  identify  hard  and  soft  lesions  in  the  prostate  tissue.  We  submitted  our  findings  and  our  paper 
has  been  accepted  for  publication  in  the  Medical  Science  Monitor  Journal  in  2012  (attached  in  Appendices): 

Ioana  Nicolaescu  Fleming,  Carmen  Kut,  Katarzina  Macura,  Li-Ming  Su,  Hassan  Rivaz,  Caitlin  Schneider, 
Ulrike  Hamper,  Tamara  Lotan,  Russ  Taylor,  Gregory  Hager,  Emad  Boctor,  "  Ultrasound  elastography  as  a  tool 
for  imaging  guidance  during  prostatectomy:  Initial  experience" ,  Med  Sci  Monit,  2012;  18(9) 


Table  2.  Prostate  specimen  data:  A  total  of  10  (ten)  elastography  lesions  were  identified  in  6  (six)  patients' specimens  (8  malignant  and  2  benign). 
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Figure  8:  Coronal  section  of  prostate  specimen  #1  at  the  level  of  the  central  gland:  Classic  ultrasound  B-mode 
(A)  and  elastogram  (B).  9.4  Tesla  ex-vivo  (C)  and  3  Tesla  in-vivo  (D)  MRI  images  are  presented  in  coronal 
planes,  in  CCW  (counter  clock  wise)  orientation  for  better  visualization  of  the  correlation  between  USE  and 
MRI  of  the  specimen.  Benign  solid  (arrow)  and  soft  (dashed  arrow)  nodules  and  urethra  are  visible. 


For  task  3c.  Automatic  robotic-assisted  palpation  (months  18-24),  we  used  our  integrated  Sonix  RP  + 
daVinci  system  (task  2a,  Figure  1).  The  assumption  was  that  automatic  pressure  application  will  produce  more 
stable  and  repeatable  images  and  thus  eliminate  inter-observer  variability  in  interpretation.  In  this 
implementation  the  tissue  is  autonomously  compressed  along  the  axial  direction  of  the  ultrasound  probe.  The 
system  computer  generates  a  sinusoidal  palpation  motion  that  is  overlaid  onto  motion  commands  from  the 
master  manipulators.  The  surgeon  retains  control  over  the  position  and  orientation  of  the  ultrasound  probe  but 
the  computer  provides  the  tremor- free,  precise  compression  motion  necessary  for  achieving  a  high  quality  strain 
image.  Continual  palpation  of  tissue  under  manual  control  is  a  tedious  task  imposing  a  large  cognitive  burden 
and  demand  of  focus.  By  relieving  this  burden  through  computer  assistance,  the  user  is  able  to  focus  on  more 
important  tasks  such  interpreting  real-time  imaging  information  and  conducting  surgery. 


After  the  automatic  palpation  motion  sequence  was  implemented,  the  system  was  tested  on  tissue  mimicking 
phantoms.  Figure  2  shows  a  view  of  the  daVinci  console  display.  The  overlay  of  the  elastogram  shows  a  clear 
differentiation  of  harder  and  softer  lesions.  The  conclusion  of  the  study  was  that  the  new  system  provides  an 


improvement  over  manual  elastography  techniques  by  unifying  motion  commands  from  a  user  with  the 
precision  and  accuracy  of  computer-assisted  motion  control  to  ensure  consistent  and  precise  tissue  strain.  This 
approach  effectively  reduces  the  cognitive  load  of  the  human  operator  while  maintaining  a  user’s  control  of  the 
procedure.  Preliminary  tests  using  an  elasticity  phantom  demonstrate  the  system’s  capability  to  generate  strain 
images  in  real-time  that  can  be  used  to  delineate  simulated  lesion  boundaries  and  differentiate  lesions  of  varying 
stiffness. 

The  results  are  reported  in  the  same  publication  attached  in  Appendices: 

Billings,  S.,  Nishikant,  D.,  Kang,  H.  J.,  Taylor,  R.,  Boctor,  E.,  " System  for  robot-assisted  real-time  laparoscopic 
ultrasound  elastography Presented  and  published  at  SPIE  Medical  Imaging  2012. 

Task  3b:  Design  and  build  new  LAPUS  probe  has  proved  unnecessary  as  our  award  progressed.  Other 
research  teams  at  Johns  Hopkins  were  also  tasked  with  evaluating  the  same  LAPUS  probe  for  other  surgical 
procedures.  The  manufacturer,  Intuitive  Surgical,  felt  the  initial  LAPUS  probe  met  the  conditions  for  successful 
deployment  in  the  clinical  setting  and  is  seeking  PDA  approval  without  further  modifications. 

Task  4.  Laparoscopic  Prostatectomy  with  Advanced  Elastography  Imaging  Methods  (months  25-33) 

Lor  task  4a:  probe  evaluation  on  phantoms,  our  study  focused  on  evaluating  the  LAPUS  probe  in  its  ability  to 
access  and  interrogate  the  surface  of  the  prostate  in  a  robotic-assisted  approach.  For  our  goal  we  employed  a 
custom  made  prostate  elastography  phantom  (CIRS,  Inc.  -  Model  066,  Figure  4).  The  phantom  was  modified 
and  its  top  was  removed  to  allow  for  LAPUS  probe  access  and  imaging  through  direct  contact  -  Figure  5. 

Phantom  Model  066  contains  3  isoechoic  lesions  that  are  at  least  two  times  stiffer  than  the  simulated  prostate 
tissue.  Under  standard  B-mode  ultrasound  they  cannot  be  detected  but  they  should  be  readily  visible  on 
elastograms  -  Figure  5. 

We  attempted  to  image  the  lesions  inside  the  phantom  using  the  LAPUS  probe  in  comparison  with  a  trans-rectal 
ultrasound  (TRUS)  probe  -  Figure  6.  The  phantom  allows  for  rectal-like  examination  and  the  TRUS  probe  is 
routinely  used  in  practice  for  imaging  the  prostate  tissue  during  biopsies  and  sometimes  during  prostatectomies. 


SIDE  VIEW 


Figure  4:  Position  of  stiffer  lesions  inside  the  Model  066  phantom. 


Figure  5:  Modified  CIRS  066  Prostate  Elastography  phantom.  The  top  has  been  removed  and  direct  access  to 
the  prostate-mimicking  tissue  was  made  available. 

The  LAPUS  probe  presented  some  challenges  we  took  note  of,  particularly  its  length  appeared  big  with  respect 
to  the  size  of  the  prostate.  If  a  custom  probe  is  to  be  built  in  the  future  just  for  prostatectomy,  it  is  our 
conclusion  that  is  should  be  shorter  than  the  current  size  of  our  LAPUS  probe.  Even  though  the  size  presented  a 
challenge,  we  were  able  to  image  a  majority  of  the  prostate  mimicking  tissue.  Palpation  with  the  probe  was 
needed  in  order  to  identify  the  stiffer  lesions  using  our  elastography  algorithm  -  Figure  7. 


Figure  6:  TRUS  probe  in  use  with  the  prostate  elastography  phantom. 


Figure  7:  Imaging  the  prostate  phantom  with  the  LAPUS  probe. 


The  analysis  of  our  comparative  elastography  study  between  the  LAPUS  and  the  TRUS  ultrasound  showed  that 
LAPUS  was  just  as  capable  as  TRUS  probe  in  identifying  the  stiffer  lesions.  One  important  thing  to  mention  is 
that  in  order  to  obtain  elastograms  with  TRUS,  we  applied  compression  on  the  tissue  from  the  top,  which  is 
equivalent  to  palpating  the  prostate  during  prostatectomy.  Classic  B-mode  ultrasound  and  elastograms  obtained 
using  TRUS  are  presented  in  Figure  9  and  the  corresponding  findings  using  LAPUS  are  shown  in  Figure  10. 
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igure  9:  Elastogram  and  classic  B-mode  ultrasound  for  3  stiff  lesions  (phantom  Model  066)  -  TRUS  imaging 
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Figure  10:  Elastogram  and  classic  B-mode  ultrasound  for  3  stiff  lesions  (phantom  Model066)  -  LAPUS  imaging 


For  task  4b:  probe  evaluation  on  animal  models,  we  intended  to  submit  for  IRB  approval  a  study  evaluating 
the  LAPUS  probe  on  the  porcine  model.  During  the  last  nine  month  of  the  award  our  team  debated  the  merit  of 
such  a  study  in  the  context  of  the  research  findings  of  2012. 

A  recent  study  at  our  institutions  was  just  publishing  their  findings  on  the  evaluation  of  the  same  LAPUS  probe 
in  an  animal  model  for  hepatic  surgery.  This  study  evaluated  a  new  integrated  ultrasonography 
(US)  device  with  the  da  Vinci  Surgical  System  for  laparoscopic  visualization  (same  LAPUS  probe  as  in  Figure 
3),  comparing  it  with  conventional  handheld  laparoscopic  ultrasound  probe  for  performing  key  tasks  in  hepatic 
surgery:  (1)  In  vivo  porcine  hepatic  visualization  and  probe  manipulation,  (2)  lesion  detection  accuracy,  and  (3) 
biopsy  precision.  Their  results  looked  favorable  for  the  robotic  probe:  the  robotic  ultrasound  probe  proved  better 
than  conventional  laparoscopic  ultrasound  in  liver  surface  exploration  (85%  success  vs  73%;  P  =  .030)  and  tool 
manipulation  (79%  vs  57%;  P  =  .028),  whereas  no  difference  was  detected  in  lesion  identification  (63  vs  58;  P  = 
.41)  and  needle  biopsy  tasks  (57  vs  48;  P  =  .1 1).  Subjects  found  the  robotic  ultrasound  probe  to  facilitate  better 
probe  positioning  (80%),  decrease  fatigue  (90%),  and  be  more  useful  overall  (90%)  on  the  post-task 
questionnaire”.  The  study  concluded  the  robot-assisted  intra-operative  ultrasound  system  to  be  practical  and 
useful  in  the  performance  of  important  tasks  required  for  hepatic  surgery,  outperforming  free-hand  laparoscopic 
ultrasound  probes  for  certain  tasks,  and  was  more  subjectively  usable  to  the  surgeon.  Two  journal  papers 
resulted  from  this  study  and  they  are  attached  as  Appendices. 

The  study  we  intended  to  submit  for  IRB  approval  intended  to  explore  similar  hypotheses  in  prostate  surgery. 
Our  team  felt  the  questions  posed  by  the  hepatic  surgery  study  were  just  as  valid  in  other  abdominal  surgeries. 
We  concluded  our  study  would  prove  repetitive  and  we  could  direct  our  effort  in  other  research  areas. 

Task  4c  proposed  the  development  of  Advanced  Ultrasound  Imaging  methods  aimed  at  improving  the  surgeons 
experience  with  ultrasound  imaging  technology.  Throughout  the  two  years  and  nine  months  of  the  award,  we 
have  made  important  contributions  to  the  ultrasound  imaging  field.  We  summarize  below  our  most  important 
research  outcomes,  as  outlined  in  the  doctoral  dissertation  of  PI  Ioana  Fleming.  The  first  four  chapters  are 
included  in  the  Appendices  (The  Thermal  Imaging  chapters  are  still  a  work  in  progress). 

Despite  ongoing  progress,  current  strain  imaging  systems  still  need  improvements  for  a  successful  adoption 
into  clinical  use.  The  medical  community  needs  a  reliable  technique  which  can  achieve  repeatable,  user 
independent,  high  strain  signal-to-noise-ratio  (SNR)  images/volumes  in  real-time.  The  displacement  estimation 
step  has  the  biggest  impact  on  the  speed  and  accuracy  of  elasticity  algorithms.  One  concern  is  applying  careful, 
optimal  compression  in  order  to  minimize  the  potential  for  global  and  local  decorrelation  between  pre-  and  post¬ 
compression  ultrasound  signals  which  usually  affects  the  signal-to-noise-ratio  (SNR).  In  in-vivo  data,  other 
sources  of  decorrelation  could  also  affect  the  computation  of  strain  images,  such  as  incoherent  fluid  (blood) 
motion,  out-of-plane  motion  of  structures  within  one  image  due  to  transducer  or  respiratory  motion,  subsample 
speckle  motion,  or  a  high  degree  of  compression. 

Thesis  Statement: 

Robust  methods  for  displacement  estimation  in  ultrasonic  strain  imaging  help  overcome  displacement 
discontinuities  and  regions  of  poorly  correlated  RF  data.  In  thermal  imaging,  estimation  of  heat-induced  echo 
strain  can  be  improved  by  detecting  and  removing  unwanted  motion  from  the  environment  which  would 
otherwise  mask  the  very  small  tissue  displacements  due  to  heating. 


Contributions: 

This  thesis  addresses  limitations  of  current  displacement  estimation  techniques  in  ultrasound  imaging.  We 
briefly  summarize  the  major  contributions  of  this  dissertation  as  follows: 

A.  Robust  Motion  Estimation  Methods  for  Ultrasound  Elastography. 

We  present  a  robust  algorithm  which  improves  on  the  dynamic  programming  2D  AM  displacement 
estimation  method  (Rivaz  et  al  2007)  in  its  ability  to  overcome  displacement  discontinuities  and  decorrelated 
regions.  Signal  decorrelation  is  widely  viewed  as  the  major  limiting  factor  for  adoption  of  ultrasound 
elastography  into  clinical  practice.  Our  contribution  is  important  as  it  increases  the  chances  of  obtaining  real¬ 
time,  repeatable,  user  independent  elastograms.  The  robust  implementation  addresses  multiple  weak  points  in 
the  Rivaz  method: 

•  By  addressing  the  selection  of  the  seed  RF  line,  our  method  insures  a  robust,  stable  starting  point  for  the 
algorithm  (Figure  11). 

•  By  controlling  the  displacement  propagation  step,  we  provide  true  displacement  in  regions  which 
otherwise  would  present  just  decorrelation  artifacts. 

We  evaluated  the  method  on  tissue  mimicking  phantoms  and  we  explore  applications  to  detection  of  cancerous 
nodules  in  the  prostate  gland  during  open  or  robotic-assisted  prostatectomy  (Figure  12). 

We  extended  the  robust  displacement  estimation  method  to  three  dimensions  by  pairing  it  with  the  frame 
selection  algorithm  of  Foroughi  et  al  (Pre-doctoral  DOD  training  award  recipient  for  breast  cancer  research). 
Combining  the  strengths  of  both  techniques  improves  the  robustness  to  signal  decorrelation.  Every  step  of  the 
method  is  highly  parallelizable.  With  the  use  of  GPU  technology,  the  method  in  its  current  form  could  be 
suitable  for  real-time  3D  elastography.  We  validated  the  3D  method  on  human  in-vivo  data  from  a  hepato- 
carcinoma  ablation. 


Integer  Displacement  Profile  for  Seed  RF-lines  249  -  255 


(a) 


(b)  (c)  (d) 

Figure  11.  Integer  DP  displacement  estimation  for  seed  RF  lines  249  -  255  (a).  Note  the  areas  (for  lines  250  and 
252  respectively)  which  exhibit  a  change  in  displacement  slope.  Strain  images  with  corresponding  artifacts  for 
seed  lines  250  (b),  251  (c),  252  (d) 


Figure  12.  Prostate  data  set  example:  (a)  an  unsuitable  seed  line  is  selected  for  subimage  II,  (b)  no  suitable  seed 
line  is  found  in  sub-image  II,  which  forces  the  algorithm  to  propagate  displacement  from  line  #35,  and  (c)  9.4 
Tesla  MRI  coronal  scan  of  the  prostate  specimen  showing  hard  lesions  (arrows),  soft  lesions  (dashed  arrows) 
and  urethra  (bracket). 


B.  Robust  Motion  Estimation  Methods  for  Thermal  Imaging 

Thermal  ablation  therapies,  using  energy  sources  like  RF,  laser,  microwave,  or  focused  ultrasound,  aim  to 
destroy  malignant  tumors  without  damaging  the  surrounding  tissue.  But  despite  promising  results,  current 
systems  remain  highly  dependent  on  operator  skill,  and  there  is  little  control  of  the  size  and  shape  of  the  ablated 
area.  Monitoring  the  spatial  distribution  of  heating  is  necessary  to  control  the  degree  of  tissue  damage  produced. 
Conventional  B-mode  ultrasound  has  been  used  historically  for  guiding  the  ablative  tools.  The  ablation  lesion 
and  surrounding  tissue  have  similar  backscatter  characteristics,  making  conventional  B-mode  ultrasound  images 
alone  unable  to  visualize  the  response  to  heating.  Many  have  observed  an  increased  echogenicity  at  the  site  of 
the  ablation  and  shadowing  below  the  thermal  lesion  (cavitation  effects).  It  has  been  concluded  that  it  is  difficult 
to  accurately  ascertain  the  size  and  position  of  the  thermal  lesion  on  B-mode  images  alone  and,  furthermore,  it 
is  nearly  impossible  to  delineate  the  treated  tissue  margins  with  conventional  ultrasound. 

Although  conventional  B-mode  ultrasound  does  not  provide  a  clear  delineation  of  the  ablated  region,  algorithms 
have  been  proposed  which  use  the  raw  RF  signals  and  are  very  similar  to  strain  estimation.  Heat-induced  echo 
strain  is  a  time-delay  estimation  problem  where  the  shifts  in  the  RF  echo  signals  are  caused  primarily  by 
variations  in  the  speed  of  sound  in  tissue  with  temperature.  A  secondary  cause  for  changes  in  the  echo  signals  is 
the  thermal  expansion  of  tissue  with  heating.  The  motion  produced  by  the  changes  in  the  speed  of  sound  is 
apparent  or  virtual,  while  thermal  expansion  of  the  tissue  introduces  an  actual  physical  shift  in  the  scatterer 
positions.  The  variations  in  the  echo  arrival  times  due  to  changes  in  the  speed  of  sound  with  temperature  are 
larger  than  the  changes  due  to  thermal  expansion  of  tissue.  As  tissue  reaches  temperatures  higher  than  50 
degrees  Celsius  and  the  speed  of  sound  in  tissue  plateaus,  thermal  expansion  effects  become  more  visible. 

The  main  challenge  in  thermal  imaging  is  measuring  very  small,  apparent  tissue  motion  (on  the  order  of  10’s  of 
microns).  Native  tissue  deformations  due  to  breathing  and  cardiac  cycle  generate  echo  and  spectral  shifts.  Other 
sources  of  movement  could  be  found  outside  the  patient’s  body,  as  even  small  vibrations  due  to  the  equipment 
in  the  surgical  suite  can  easily  mask  the  very  small  temperature-induced  shifts.  In  this  thesis  we  develop  a 
robust  method  for  heat-induced  echo  strain  estimation  in  thermal  imaging.  An  iterative  motion  compensation 
algorithm  is  developed  which  can  detect  and  alter  displacement  of  tissue  due  to  other  sources  of 
deformation/movement.  Estimating  temperature-induced  echo  shifts  is  a  very  difficult  problem  because  the 
apparent  movement  can  be  very  small,  on  the  order  of  tens  of  microns.  We  show  that  removing  extraneous 
motion  helps  unmask  the  effects  of  heating  (Figure  13).  The  method  is  evaluated  on  ablated  lesions  in  ex-vivo 
tissue. 


The  Iterative  Compensation  algorithm  is  summarized  below: 


1.  One  RF  frame  is  selected  from  the  1st  data  point  and  one  RF  frame  from  the  last  data  point.  Estimate 
displacement  between  the  two  RF  frames  using  the  robust  algorithm  (A).  Compute  strain  data. 

2.  Create  mask  for  the  region  least  affected  by  the  heating.  The  masks  use  a)  5%,  b)  10%  or  c)  15%  lowest 
axial  and  lateral  strain  areas.  The  resulting  masks  for  two  data  sets  are  presented  in  Figure  13. 
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Figure  13:  Masks  for  the  regions  least  affected  by  the  heating.  The  masks  use  a)  5%,  b)  10% 

or  c)  15%  lowest  axial  and  lateral  strain  areas.  First  row  corresponds  to  the  handheld  data  set,  second  row  to  the 

fixed,  robot  held  data  set. 


3.  Iterate  back  to  intermediate  data  points.  Compute  motion  (axial  and  lateral  displacement)  in  the  areas 
masked.  Subtract  motion  (median  value)  to  achieve  compensated  motion  field  (Figure  14). 

4.  Compare  new  strain  images  with  old  ones  (Figure  15). 

5.  Select  ROIs  (Figure  19) 

6.  Compute  min,  max,  mean  lateral  strain  for  each  ROI.  Compare  the  original  values  with  the  new  values 
after  IMC  (iterative  motion  compensation)  algorithm  (Figure  20  -  Fig.  7,  8,  9  selected  from  poster 
presentation). 

We  validated  our  algorithm  on  two  data  sets  acquired  on  ablated  ex-vivo  tissue.  In  the  first  data  set  the 
ultrasound  probe  was  hand  held,  while  in  the  second  data  set  the  transducer  was  fixed,  supported  by  a  robotic 
arm.  Figure  14  shows  the  uncovered  additional  motion  during  the  heating  process.  The  hand  held  data  is 
obviously  corrupted  by  the  motion  of  the  operator’s  hand  (physiological  tremor).  In  contrast,  the  motion 
uncovered  in  the  data  set  with  the  fixed  robot-supported  probe,  in  three  orders  of  magnitude  smaller  and  the 
three  masks  employed  uncovered  very  similar  results  (Figure  14,  a  versus  b). 

Our  results  show  the  IMC  approach  improves  the  heat  induced  echo-strain  images.  The  ablated  region  is 
represented  more  clearly  in  the  strain  images  after  the  IMC  algorithm  was  applied  (Figure  15).  The  size  and 
shape  of  the  ablated  region  is  in  alignment  with  the  gross  pathology  findings  (Figure  16).  Elastographic  signal 
to  noise  ratio  (SNR)  remains  approximately  constant  after  the  IMC  approach  (figure  17),  while  the  contrast  to 
noise  ratio  (CNR)  is  shown  to  greatly  improve,  regardless  of  the  mask  used  in  IMC  (figure  1 8) 
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Figure  14:  Motion  vectors  uncovered  by  the  IMC  algorithm,  a)  hand-held  probe,  b)  robot  supported,  fixed  probe 
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Figure  15:  Axial  echo  strain  induced  by  tissue  heating.  First  row  shows  the  original  computed  axial  strain  at 
each  step  of  the  heating  process.  The  last  three  rows  show  the  axial  strain  obtained  with  the  Iterative  Motion 
Compensation  algorithm,  using  respectively  a  5%  mask  (A),  10%  (B),  and  15%  (C). 


Figure  16:  Gross  pathology  images  of  ablated  lesion  in  hand-held  probe  experiment  (left)  and  robot-supported 
probe  experiment  (right). 


Figure  17:  Elastographic  SNR  of  resulting  axial  (left)  and  lateral  (right)  strain:  Original  computation  versus 
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Figure  18:  Elastographic  CNR  of  resulting  axial  (left)  and  lateral  (right)  strain:  Original  computation  versus 
after  IMC  algorithm. 
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Figure  19:  Regions  of  interest  selected  for  comparison  of  strain  values  before  and  after  IMC  algorithm  (green). 
Red  ROI  represents  the  location  of  the  heating  element.  Blue  ROI  represents  the  location  of  the  temperature 
reading  probe.  Hand-held  probe  experiment  (left)  and  robot-supported  probe  experiment  (right). 


Fig.  7:  Mean  lateral  displacement  in  outer 
regions  of  interest  (blue),  before  and  after  IMC 
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Fig.  8:  Mean  lateral  displacement  in  inner 
regions  of  interest  (green),  before  and  after  IMC 


Fig.  9:  Mean  lateral  displacement  in  outer  vs.  inner 
regions  of  interest  (blue  vs.  green),  before  and  after  IMC 


Figure  20:  Examples  of  lateral  displacement  values  in  the  selected  ROIs  for  robot-supported,  fixed  probe 
experiment. 


C.  Ultrasound  Elastography  Mosaicking 


Compared  to  other  imaging  modalities  like  CT  and  MR,  ultrasound  suffers  from  a  limited  field  of  view  (FOV). 
Monitoring  a  structure  can  be  particularly  challenging  when  it  is  too  large  to  be  visualized  in  a  single  image  or 
3D  sweep.  Size  and  distance  measurements  are  unreliable  in  large  organs.  With  the  ability  to  obtain  a  3D 
ultrasound  volume,  the  next  evolutionary  step  in  3D  ultrasound  is  to  create  an  extended  field  of  view  by 
stitching  several  volumes  together.  Panoramic  ultrasound  imaging  is  emerging  as  a  prevalent  technique  in 
clinical  practice  with  a  high  clinical  value.  Also  referred  to  as  stitching  or  panorama,  ultrasound  mosaics  aim  to 
achieve  several  clinical  advantages  which  come  along  with  extended  FOV:  1)  improving  the  understanding  of 
spatial  relationships  among  structures  when  the  size  of  a  single  image/volume  is  not  large  enough  to  cover  the 
entire  region  of  interest,  2)  visualizing  structures  that  are  too  large  for  a  single  volume  and  3)  allowing  for 
measurements  of  size  and  distance  in  large  organs  and  lesions,  and  4)  compounded  volumes  of  higher  quality 
will  offer  the  ability  to  visualize  the  anatomical  structures  from  a  variety  of  angles. 

In  the  literature  of  ultrasound  mosaicking,  registering  the  underlying  displacement  field  for  elastography  has  not 
yet  been  addressed.  The  clinical  advantages  of  ultrasound  mosaics  can  be  improved  with  the  additional 
corresponding  strain  information.  In  particular,  strain  information  can  help  in  multi-modal  registration  and 
fusion  with  pre-operative  data  for  guidance  in  minimally-invasive  interventions.  In  this  thesis,  we  propose  a 
technique  for  generating  a  reliable,  wide  field-of-view  displacement  field,  robust  to  sources  of  decorrelation. 
Elastography  mosaics  are  generated  from  two  pairs  of  ultrasound  images,  and  then  from  multiple  image  pairs. 
The  method  is  extended  to  3D  ultrasound  elastography  mosaicking  using  multiple  3D  volume  pairs. 

We  develop,  to  the  best  of  our  knowledge,  the  first  algorithm  for  generating  reliable  multi-image  ultrasound 
elastography  mosaics,  robust  to  regions  of  decorrelation.  For  proof-of-concept,  the  method  is  evaluated  on  2D 
and  3D  data  from  tissue-mimicking  phantoms  (Figure  21). 
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Figure  21:  Tissue  mimicking  phantom:  ultrasound  elastography  at  position  tO  (a),  tl  (b),  t2  (c),  t3  (d)  and 
elastography  mosaic  of  positions  tO  and  tl  (e),  tO,  tl  and  t2  (f),  and  tO,  tl,  t2  and  t3  (g). 


3.  KEY  RESEARCH  ACCOMPLISHMENTS 


•  Integration  of  an  existing  laparoscopic  ultrasound  probe  with  the  daVinci  robotical  surgical  system. 

•  Testing  of  the  new  integrated  system  on  tissue  mimicking  phantoms. 

•  Automatic  robotic-assisted  palpation:  design,  implementation  and  testing. 

•  Completion  of  study  on  ex-vivo  prostate  specimens  using  standard  ultrasound  probe  as  an  equivalent  to 
the  LAPUS  probe. 

•  Analysis  of  study  data;  results  show  elastography  has  the  potential  to  detect  prostate  cancer  through 
direct  interrogation  techniques. 

•  Phantom  study  using  a  modified  prostate  elastography  phantom;  comparison  of  LAPUS  and  TRUS; 
results  show  feasibility  in  lesion  detection  with  elastography  but  there  are  concerns  about  probe 
manipulation  due  to  its  size. 

•  Development  of  a  robust  displacement  estimation  method  (2D  and  3D)  which  improved  the  ability  to 
overcome  areas  of  signal  decorrelation.  The  method  was  validated  on  tissue  mimicking  phantoms, 
prostate  ex-vivo  specimens  and  in-vivo  porcine  ablated  tissue. 

•  Development  of  a  robust  iterative  motion  compensation  algorithm  for  strain  induced  thermal  imaging. 
The  algorithm  was  validated  on  ex-vivo  ablated  tissue. 
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Other  achievements: 

The  PI  has  developed  and  served  as  the  main  instructor  in  a  Johns  Hopkins  Intersession  course  entitled 
“Introduction  to  Medical  Imaging”.  For  four  years,  approximately  25  Hopkins  undergraduate  enrolled  every 
Winter  Intersession  in  Ioana  Fleming’s  course.  The  course  became  very  popular  and  seats  were  frequently 
completely  occupied  the  day  enrollment  opened  for  the  session.  Feedback  from  students  as  expressed  in  written 
evaluations  has  been  overwhelmingly  positive.  In  addition  to  teaching  about  the  physics  and  applications  of 
various  medical  imaging  technologies,  the  PI  has  used  the  course  to  raise  awareness  for  Prostate  Cancer  and  its 
treatments,  from  minimally  invasive  surgery  to  ultrasound  elastography. 


5.  CONCLUSION 


The  work  completed  under  the  sponsorship  of  the  DOD  award  has  advanced  research  in  prostate  cancer 
therapies.  PI  will  defend  PhD  dissertation  in  the  Fall  of  2013. 
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Chapter  1 

Introduction 


1.1  Ultrasound  Imaging 

The  field  of  medical  imaging  is  advancing  at  a  rapid  pace.  Medical  imaging  modalities 
like  x-ray  radiography,  computed  tomography  (CT),  nuclear  imaging  and  magnetic 
resonance  imaging  (MRI),  are  being  used  in  laboratories  and  hospitals  today  not  only 
to  visualize  anatomical  structures,  but  also  to  detect  diseases  and  plan  out  surgical 
interventions.  Ultrasound  imaging  stands  alone  as  the  cheapest  and  safest  imaging 
modality.  It  is  also  the  world’s  most  frequently  used  imaging  modality,  with  the 
largest  number  of  images  created  annualy  [1] . 
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Sound  waves  are  the  organized  vibrations  of  atoms  or  molecules  in  tissue.  The  audible 
range  of  human  ear  is  from  20  Hz  to  20  kHz.  Ultrasound  represents  sound  waves 
generally  above  20  kHz,  with  medical  ultrasound  usually  in  the  1  to  10  MHz  range. 
The  basic  principle  of  ultrasound  imaging  can  be  described  as  follows  [1]: 

An  ultrasound  pulse  is  transmitted  into  tissue  where  it  encounters  interfaces  and  scat¬ 
ters  that  produce  echoes  ( reflections )  as  the  pulse  penetrates  tissue.  The  echoes  are 
recorded  and  their  magnitude  and  timing  are  used  to  form  an  image. 

The  component  that  generates  the  pulses  and  measures  the  echoes  is  a  piezoelectric 
crystal.  Piezoelectric  crystals,  often  refered  to  as  transducers,  can  be  used  for  ul¬ 
trasound  transmission  and  reception.  They  are  the  interface  between  the  electronic 
circuits  of  an  ultrasound  machine  and  the  physical  world. 

In  ultrasound  A-mode  imaging,  a  transducer  with  a  single  piezoelectric  element  is 
excited  with  a  voltage  spike  to  transmit  a  pulse  into  tissue.  As  the  pulse  travels 
into  the  tissue,  it  generates  echos  from  the  reflections  at  tissue  boundaries.  The 
returning  echos  generate  voltage  spikes  which  are  recorded  and  digitized.  We  refer  to 
the  raw  unprocessed  array  of  electric  signals  as  the  radio- frequency  (RF)  signal.  The 
total  time  between  the  initial  pulse  and  the  echo  is  proportional  to  the  depth  of  the 
boundary,  while  the  amplitude  of  the  echo  is  proportional  to  the  difference  in  acoustic 
impedances  at  the  tissue  boundaries.  These  two  pieces  of  information,  coupled  with 
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Figure  1.1:  Illustration  of  A-mode  ultrasound  imaging  and  the  pulse-echo  technique 
(J.  L.  Prince  and  J.  Links,  Medical  Imaging  Signals  and  Systems.  Prentice  Hall,  2005) 


a  known  or  approximate  value  for  the  speed  of  sound  in  the  imaged  tissue,  allow  a 
one-dimensional  mapping  of  the  tissue  boundaries  along  the  line  of  propagation  of  the 
pulse.  The  pulse-echo  technique  of  A-mode  imaging  produces  ID  signals  (Fig.  1.1), 
where  the  amplitude  of  the  echoes  are  displayed  to  the  user  as  a  function  of  depth. 


B-mode  imaging  is  the  natural  extension  of  A-mode  imaging  into  2D.  A  modern 
ultrasound  hand-held  probe  contains  a  line  of  piezoelectric  elements.  Instead  of  dis¬ 
playing  the  amplitude  versus  distance,  here  brightness  is  used,  hence  the  B  in  B-mode. 
The  RF  signals  are  combined  in  columns  and  the  resulting  2D  image  contains  pixels 
of  brightness,  calculated  as  the  logarithmically  scaled  envelope  of  the  RF  signal  at 
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each  depth/location. 


1.2  Motion  Estimation  in  Ultrasound  Im¬ 
ages 

Research  developments  have  transformed  ultrasonography  from  a  qualitative  tool  for 
imaging  soft  tissue  to  a  quantitative  tool  for  measuring  tissue  properties.  In  par¬ 
ticular,  two  cases  have  seen  increased  interest  in  recent  years:  1)  measurement  of 
tissue  stiffness  (known  as  strain  imaging  or  elasticity  imaging),  and  2)  measurement 
of  change  in  tissue  temperature  (known  as  thermal  imaging).  Both  elastography  and 
thermal  imaging  rely  fundamentally  on  the  measurement  of  real  or  apparent  motion 
in  ultrasound  image  sequences. 


1.2.1  Ultrasound  Elastography 

Physicians  have  used  palpation  for  thousands  of  years  and  for  some  diseases  it  is 
still  the  primary  diagnostic  tool.  Examples  include  breast  self-examination  for  early 
detection  of  breast  lumps  and  digital  rectal  examination  for  prostate  cancer  nodules. 
The  reason  palpation  is  so  effective  as  a  medical  tool  is  based  on  the  fact  that  many 
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diseases  cause  changes  in  tissue  mechanical  properties.  Various  inflamatory  and  neo¬ 
plastic  processes  result  is  an  increase  in  stiffness  or  elastic  modulus  of  the  tissue.  It 
is  also  worth  noting  that  surgeons  routinely  use  manual  palpation  intra-operatively 
to  asses  the  degree  of  stiffness  in  tissues  undetected  preoperatively  by  CT,  MRI,  or 
conventional  B-mode  ultrasound. 

Several  years  ago  Ophir  et  al  [2]  introduced  a  new  method  termed  elastography  for 
direct  ultrasound  imaging  of  the  stiffness  of  tissues.  Emulating  manual  palpation,  the 
tissue  was  compressed  and  apparent  motion  of  regions  of  the  image  were  compared. 
From  this  information,  measurements  related  to  tissue  stiffness  were  computed. 

The  main  components  of  elasticity  imaging  are: 

1.  Data  capturing  during  externally  or  internally  applied  tissue  motion  or  defor¬ 
mation, 

2.  Evaluation  of  tissue  response,  and, 

3.  Reconstruction  of  the  elastic  modulus  or  other  physical  properties  based  on  the 
theory  of  elasticity. 

Static  elasticity  methods  involve  applying  compression  directly  to  the  tissue  [2,3,4], 
while  dynamic  ones  induce  motion  by  applying  a  low-frequency  vibration  to  the  tissue 
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[5,6,7,  8,9].  Many  literature  reviews  describe  various  dynamic  and  static  elasticity 
imaging  techniques  [10, 11,12, 13, 14] .  In  both  methods,  RF  signals  are  acquired  before 
and  after  tissue  excitation  and  are  then  used  to  estimate  tissue  motion.  For  some 
elasticity  methods  the  final  resulting  image  maps  physical  parameters,  like  Youngs 
modulus  and  Poissons  ratio,  while  others  provide  a  more  qualitative  image  of  stiffness 
distribution  in  tissue.  The  literature  refers  to  measurements  acquired  under  static  and 
dynamic  excitation  collectively  as  elasticity  imaging.  Medical  applications  range  from 
tumor  detection  [15],  to  characterization  of  vascular  plaques  [16]  and  assessment  of 
vascular  health  [17]  to  the  study  of  skeletal  muscle  contraction  [18],  assessment  of 
fetal  lung  maturity  [19],  and  renal  transplant  rejection  [20]. 

Among  the  techniques  used  for  tissue  displacement  estimation  in  quasi-static  elastog- 
raphy,  two  have  been  studied  in  depth:  correlation  based  approaches  and  phase-based 
methods.  The  method  as  first  proposed  by  Ophir  et  al  [2]  used  correlation  maximiza¬ 
tion  between  windows  of  RF  signal  from  the  pre  and  post-deformation  images.  It  still 
remains  the  most  widely  used  method  today.  Local  axial  strain  is  estimated  from  the 
A-line  motion  as  [21]: 


^11, local 


(tlb  ~  tig)  ~  j^2b  ~  tj, a) 
lib  tla 


(1.1) 


where  tla  and  t\i}  represent  the  arrival  times  of  the  pre-compression  echo  from  the 
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distal  and  proximal  windows,  and  t2a  and  t2b  the  corresponding  arrival  times  of  the 
post-compression  echo.  The  reference  region  is  translated  in  small  overlapping  steps 
along  the  RF  echo  signal  and  one  measure  of  tissue  motion  is  calculated  for  each 
depth.  The  strain  image  produced  by  displaying  spatial  derivatives  from  the  estimated 
displacement  field  was  called  an  elastogram. 

The  second  dominant  approach  for  tissue  displacement  estimation  was  pioneered  by 
ODonnell  et.  al.  [3,4],  who  showed  that  there  is  a  direct  relationship  between  the 
relative  displacement  between  two  windows  and  the  phase  of  the  complex  correlation 
between  baseband  signal  windows.  At  time  t,  the  baseband  representation  of  two 
signals  £i(f)  and  x2 (t)  are  [3,4]: 


Xi(t)  =  A{t  - 

(1.2) 

x2(t)  =  A(t  -  T2)e~luJoT2, 

(1.3) 

where  A  is  the  real  envelope  of  the  transceived  pulse,  cuq  is  the  angular  frequency  of 
the  ultrasound  carrier,  and  is  the  round-trip  propagation  time  from  the  transducer 
to  the  point  scatterer.  The  complex  correlation  function  between  the  two  signals  can 
be  evaluated  as: 


C(t)  = 


X1X2  (t  +  r)d,T  =  A{t  —  T\)A{t  —  r2)e 


-io>o(T2-Tl) 


(1.4) 
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The  time  delay  between  x\ (t)  and  x2(t)  can  be  estimated  from  the  zero  phase  of 
C,  while  a  non-zero  phase  can  be  used  to  estimate  the  location  of  the  peak  without 


further  search  [4]: 


A</>(0)  =  tan  1 


Im(x(  0)) 
i?e(x(0)) 


Arc 


c  *  A  <f> 
2(jJq 


(1.5) 

(1.6) 


where  c  is  the  speed  of  sound. 


Despite  ongoing  progress,  current  strain  imaging  systems  still  need  improvements 
for  a  successful  adoption  into  clinical  use.  The  medical  community  needs  a  reliable 
technique  which  can  achieve  repeatable,  user  independent,  high  strain  signal-to-noise- 
ratio  (SNR)  images/ volumes  in  real-time.  The  displacement  estimation  step  has  the 
biggest  impact  on  the  speed  and  accuracy  of  elasticity  algorithms.  One  concern  is 
applying  careful,  optimal  compression  in  order  to  minimize  the  potential  for  global 
and  local  decorrelation  between  pre-  and  post-compression  ultrasound  signals  which 
usually  affects  the  signal-to- noise-ratio  (SNR).  In  in-vivo  data,  other  sources  of  decor¬ 
relation  could  also  affect  the  computation  of  strain  images,  such  as  incoherent  fluid 
(blood)  motion,  out-of-plane  motion  of  structures  within  one  image  due  to  transducer 
or  respiratory  motion,  subsample  speckle  motion,  or  a  high  degree  of  compression. 
In  window/block  matching  methods,  decorrelation  introduces  significant  noise  into 
the  displacement  map.  Correlation-based  methods  cannot  tollerate  large  tissue  de¬ 
formations  which  would  be  desirable  for  high  SNR  [21,22, 12].  Phase  zero  estimation 
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methods  also  require  an  estimate  of  the  center  frequency  of  the  ultrasound  RF  signal, 
which  varies  with  depth  due  to  frequency-dependent  attenuation  in  tissue  [23]. 

Many  enhancements  to  the  displacement  estimation  step  have  been  proposed  to  re¬ 
duce  signal  decorrelation  effects,  most  notably  signal  companding  (compression  and 
expanding)  [24,22,25].  Axial  stretching  can  only  compensate  for  decorrelation  due 
to  scatterer  motion  in  the  axial  direction;  decorrelation  due  to  lateral  and  eleva- 
tional  motion  or  other  sources  do  not  benefit  from  these  methods.  In  addition,  there 
has  been  active  research  in  improving  the  quality  of  strain  estimation.  Tracking  ap¬ 
proaches  have  focused  on  the  way  the  displacement  map  is  accumulated.  Assuming 
displacement  continuity,  Pesavento  et  al.  [26]  and  Zhu  et  al.  [27]  were  the  first  to 
propose  elastography  algorithms  where  each  point’s  displacement  is  initialized  to  the 
previously  calculated  displacement  value  of  it’s  neighbor.  Other  row-to-row  track¬ 
ing  algorithms  have  been  proposed  [28]  and  similar  techniques  have  been  developed 
for  column  [29,30],  diagonal  [31]  or  other  [28,32]  directions  of  propagation.  Chen 
et  al.  [33]  rejected  the  displacement  continuity  assumption  and  implemented  an  al¬ 
gorithm  governed  by  the  presence  of  high-quality  data  areas,  where  displacement  is 
propagated  around  these  regions  first  and  poorly  correlated  data  segments  are  esti¬ 
mated  last  to  limit  their  overall  influence. 

In  general,  these  post-processing  steps  improve  signal  quality,  but  they  also  introduce 
significant  computation  demands,  making  it  challenging  to  display  elastograms  in 
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real-time.  The  ability  to  display  the  strain  map  in  real-time  to  the  user  is  very  valu¬ 
able  in  the  medical  community,  where  such  capability  greatly  increases  the  potential 
elastography  might  become  the  tool  of  choice  for  image  guided  surgical  interventions. 
The  speed  of  the  displacement  estimation  becomes  even  more  important  in  three- 
dimensional  elasticity  imaging,  where  the  volume  of  the  data  increases  computational 
demands.  Jiang  el  al  [30]  and  Rivaz  et  al.  [34]  were  the  first  to  frame  the  displace¬ 
ment  estimation  as  an  optimization  problem  solved  by  dynamic  programming.  Rivaz 
et  al  [35]  greatly  improved  the  speed  of  the  displacement  estimation  without  sacrificing 
accuracy.  The  2D  Analytic  Minimization  (AM)  method  uses  dynamic  programming 
to  estimate  2D  sub-pixel  displacement  values  on  one  seed  axial  RF  line,  which  are 
later  propagated  laterally  throughout  the  entire  image. 

First  3D  ultrasound  imaging  systems  used  ID  arrays  being  mechanically  translated  in 
the  elevational  direction  [36] .  A  similar  technique  was  adopted  in  3D  strain  imaging, 
with  additional  small  variations  in  the  applied  compression  between  adjacent  frame 
pairs  [37,38].  Another  technique  for  3D  strain  imaging  is  to  acquire  the  pre-  and  post¬ 
compression  volumes  sequentially.  A  small  pressure  is  applied  to  the  tissue  between 
volume  acquisitions  using  a  mechanical  or  simulated  fixture  [39,40,41,42,43,44],  or 
by  freehand  compression  between  volume  acquisitions  [32],  These  methods  are  slow 
and  cumbersome  in  the  data  acquisition  phase,  requiring  great  care  to  avoid  signal 
decorrelation  between  the  pre-  and  post-compression  frames. 
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Two  hybrid  approaches  stand  out  recently.  Houdsen  et  al.  [45]  applied  freehand 
compression  at  each  step  of  the  sweep  and  selected  several  frames,  then  the  stepper 
motor  advanced  to  the  next  location  in  the  elevational  direction.  This  method  has  ease 
of  acquisition  as  it  bypasses  the  practical  difficulty  of  performing  careful  and  optimal 
compression  between  pre-  and  post-compression  frames.  It  still  suffers  however  from 
the  point  of  view  of  data  acquisition,  which  can  be  quite  slow  for  large  volumes. 
Tracking  the  2D  wobbler  transducer  using  a  position  sensor,  allows  Foroughi  et  al. 
to  select  aligned  pairs  of  RF  frames  with  minimal  out-of-plane  motion,  ensuring  high 
quality  strain  [46,?].  The  acquisition  is  fast  as  RF  data  is  continuously  collected  while 
the  operator  performs  freehand  compression  and  relies  on  the  tracking  algorithm  to 
select  suitable  pairs. 

In  this  thesis  we  develop  an  algorithm  which  improves  on  the  robustness  and  accuracy 
of  the  2D  AM  displacement  estimation  method  [35]  and  extends  it  to  three  dimen¬ 
sions.  This  method  enhances  the  dynamic  programming  approach  to  displacement 
estimation,  improving  its  ability  to  overcome  displacement  discontinuities  and  regions 
of  poorly  correlated  RF  data.  Pairing  the  robust  methods  with  the  frame  selection 
algorithm  [?]  combines  the  strenghts  of  both  techniques  and  allows  for  fast  displace¬ 
ment  estimation  for  3D  data.  Every  step  of  the  algorithm  is  highly  paralallelizable, 
thus  greatly  decreasing  the  computational  time. 
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1.2.2  Thermal  Imaging 

Thermal  ablation  therapies,  using  energy  sources  like  RF,  laser,  microwave,  or  focused 
ultrasound,  aim  to  destroy  malignant  tumors  without  damaging  the  surrounding  tis¬ 
sue.  But  despite  promising  results,  current  systems  remain  highly  dependent  on 
operator  skill,  and  there  is  little  control  of  the  size  and  shape  of  the  ablated  area. 
Monitoring  the  spatial  distribution  of  heating  is  necessary  to  control  the  degree  of 
tissue  damage  produced.  Conventional  B-mode  ultrasound  has  been  used  histori¬ 
cally  for  guiding  the  ablative  tools.  The  ablation  lesion  and  surrounding  tissue  have 
similar  backscatter  characteristics,  making  conventional  B-mode  ultrasound  images 
alone  unable  to  visualize  the  response  to  heating.  Many  have  observed  an  increased 
echogenity  at  the  site  of  the  ablation  and  shadowing  below  the  thermal  lesion  (cavita¬ 
tion  effects)  [47,48].  It  has  been  concluded  that  it  is  difficult  to  accurately  ascertain 
the  size  and  position  of  the  thermal  lesion  on  B-mode  images  alone  and,  furthermore, 
it  is  nearly  impossible  to  delineate  the  treated  tissue  margins  with  conventional  ul¬ 
trasound. 

Although  conventional  B-mode  ultrasound  does  not  provide  a  clear  delineation  of  the 
ablated  reagion,  algorithms  have  been  proposed  which  use  the  raw  RF  signals  and  are 
very  similar  to  strain  estimation.  Heat-induced  echo  strain  is  a  time-delay  estimation 
problem  where  the  shifts  in  the  RF  echo  signals  are  caused  primarily  by  variations 
in  the  speed  of  sound  in  tissue  with  temperature.  A  secondary  cause  for  changes  in 


12 


CHAPTER  1.  INTRODUCTION 


the  echo  signals  is  the  thermal  expansion  of  tissue  with  heating  [49].  The  motion 
produced  by  the  changes  in  the  speed  of  sound  is  apparent  or  virtual,  while  thermal 
expansion  of  the  tissue  introduces  an  actual  physical  shift  in  the  scatterer  positions. 
The  variations  in  the  echo  arrival  times  due  to  changes  in  the  speed  of  sound  with 
temperature  are  larger  than  the  changes  due  to  thermal  expansion  of  tissue  [49, 50] . 
As  tissue  reaches  temperatures  higher  than  50 °C  and  the  speed  of  sound  in  tissue 
plateaus,  thermal  expansion  effects  become  more  visible  . 

Several  techniques  are  based  on  estimating  the  echo-shift  using  speckle  tracking  [49, 
50,51,52,53],  and  differentiating  the  time-shift  estimates  along  the  axial  direction 
to  obtain  a  temperature  map.  Temperature  estimates  are  obtained  using  a  cross¬ 
correlation  algorithm  applied  to  raw  ultrasound  RF  data  acquired  at  discrete  intervals 
during  heating  [47] .  Measurements  were  validated  experimentally  in  tissue  and  tissue- 
mimicking  samples  [48, 54, 55, 56] . 

The  main  challenge  in  thermal  imaging  is  measuring  very  small,  apparent  tissue 
motion  (on  the  order  of  10s  of  microns).  Native  tissue  deformations  due  to  breathing 
and  cardiac  cycle  generate  echo  and  spectral  shifts.  Other  sources  of  movement  could 
be  found  outside  the  patients  body,  as  even  small  vibrations  due  to  the  equipment  in 
the  surgical  suite  can  easily  mask  the  very  small  temperature- induced  shifts.  In  this 
thesis  we  develop  a  robost  method  for  thermal  strain  estimation  based  on  the  dynamic 
programming  2D  AM  technique  by  Rivaz  et  al  [35].  The  robustness  is  obtained  by 
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developing  an  iterative  motion  compensation  algorithm  which  can  detect  and  remove 
displacement  of  tissue  due  to  changes  in  the  environment. 


1.2.3  Ultrasound  Elastography  Mosaicking 

Compared  to  other  imaging  modalities  like  CT  and  MR,  ultrasound  suffers  from  a 
limited  field  of  view  (FOV).  Monitoring  a  structure  can  be  particularly  challenging 
when  it  is  too  large  to  be  visualized  in  a  single  image  or  3D  sweep.  Size  and  dis¬ 
tance  measurements  are  unreliable  in  large  organs.  With  the  ability  to  obtain  a  3D 
ultrasound  volume,  the  next  evolutionary  step  in  3D  ultrasound  is  to  create  an  ex¬ 
tended  field  of  view  by  stitching  several  volumes  together.  Panoramic  ultrasound 
imaging  is  emerging  as  a  prevalent  technique  in  clinical  practice  with  a  high  clinical 
value.  Also  referred  to  as  stitching  or  panorama,  ultrasound  mosaicks  aim  to  achieve 
several  clinical  advantages  which  come  along  with  extended  FOV:  1)  improving  the 
understanding  of  spatial  relationships  among  structures  when  the  size  of  a  single  im¬ 
age/volume  is  not  large  enough  to  cover  the  entire  region  of  interest,  2)  visualising 
structures  that  are  too  large  for  a  single  volume  and  3)  allowing  for  measurements  of 
size  and  distance  in  large  organs  and  lesions,  and  4)  compounded  volumes  of  higher 
quality  will  offer  the  ability  to  visualise  the  anatomical  structures  from  a  variety  of 
angles. 
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In  the  literature  of  ultrasound  mosaicing,  registering  the  underlying  displacement  field 
for  elastography  has  not  yet  been  addressed.  The  clinical  advantages  of  ultrasound 
mosaics  can  be  improved  with  the  additional  corresponding  strain  information.  In 
particular,  strain  information  can  help  in  multi-modal  registration  and  fusion  with 
pre-operative  data  for  guidance  in  minim  ally-invasive  interventions.  In  this  thesis,  we 
propose  a  technique  for  generating  a  reliable,  wide  field-of-view  displacement  field, 
robust  to  sources  of  decorrelation.  Elastography  mosaics  are  generated  from  two  pairs 
of  ultrasound  images,  and  then  from  multiple  image  pairs.  The  method  is  extended 
to  3D  ultrasound  elastography  mosaicing  using  multiple  3D  volume  pairs. 


1.3  Thesis  Statement 

Robust  methods  for  displacement  estimation  in  ultrasonic  strain  imaging  help  over¬ 
come  displacement  discontinuities  and  regions  of  poorly  correlated  RF  data.  In  ther¬ 
mal  imaging,  estimation  of  heat-induced  echo  strain  can  be  improved  by  detecting 
and  removing  unwanted  motion  from  the  environment  which  would  otherwise  mask 
the  very  small  tissue  displacements  due  to  heating. 
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1.4  Contributions 

This  thesis  addresses  limitations  of  current  displacement  estimation  techniques  in 
ultrasound  imaging.  We  briefly  summarize  the  major  contributions  of  this  dissertation 
as  follows: 

•  We  present  a  robust  algorithm  which  improves  on  the  dynamic  programming  2D 
AM  displacement  estimation  method  [35],  in  its  ability  to  overcome  displace¬ 
ment  discontinuities  and  decorrelated  regions.  Signal  decorrelation  is  widely 
viewed  as  the  major  limiting  factor  for  adoption  of  ultrasound  elastography 
into  clinical  practice.  Our  contribution  is  important  as  it  increases  the  chances 
of  obtaining  real-time,  repeatable,  user  independent  elastograms.  The  robust 
implementation  addresses  multiple  weak  points  in  the  Rivaz  method  [35]: 

—  By  addressing  the  selection  of  the  seed  RF  line,  our  method  insures  a 
robust,  stable  starting  point  for  the  algorithm. 

—  By  controlling  the  displacement  propagation  step,  we  provide  true  displace¬ 
ment  in  regions  which  otherwise  would  present  just  decorrelation  artefacts. 

We  evaluate  the  method  on  tissue  mimicking  phantoms  and  we  explore  appli¬ 
cations  to  detection  of  cancerous  nodules  in  the  prostate  gland  during  open  or 
robotic-assisted  prostatectomy. 
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•  We  extend  the  robust  displacement  estimation  method  to  three  dimensions  by 
pairing  it  with  the  frame  selection  algorithm  of  Foroughi  et  al  [57] .  Combining 
the  strenghts  of  both  techniques  improves  the  robustness  to  signal  decorrela¬ 
tion.  Every  step  of  the  method  is  highly  parallelizable.  With  the  use  of  GPU 
technology,  the  method  in  its  current  form  could  be  suitable  for  real-time  3D 
elastograpy.  We  validated  the  3D  method  on  human  in-vivo  data  from  a  hepato- 
carcinoma  ablation. 

•  We  develop  a  robust  method  for  heat-induced  echo  strain  estimation  in  ther¬ 
mal  imaging.  An  iterative  motion  compensation  algorithm  is  developed  which 
can  detect  and  filter  displacement  of  tissue  due  to  other  sources  of  deforma¬ 
tion/movement.  Estimating  temperature-induced  echo  shifts  is  a  very  difficult 
problem  because  the  apparent  movement  can  be  very  small,  on  the  order  of 
tens  of  microns.  We  show  that  removing  extraneous  motion  helps  unmask  the 
effects  of  heating.  The  method  is  evaluated  on  ablated  lesions  in  ex-vivo  tissue. 

•  We  develop,  to  the  best  of  our  knowledge,  the  first  algorithm  for  generating  reli¬ 
able  multi-image  ultrasound  elastography  mosaics,  robust  to  regions  of  decorre¬ 
lation.  For  proof-of-concept,  the  method  is  evaluated  on  2D  and  3D  data  from 
tissue-mimicking  phantoms. 
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1.5  Outline 

In  Chapter  2  we  focus  on  the  robust  displacement  estimation  method  for  ultrasound 
elastography.  We  first  introduce  the  reader  to  the  regularized  dynamic  programming 
displacement  estimation  technique  by  [35].  Methods  are  presented  for  robust  seed 
RF  line  selction  and  for  correlation-guided  displacement  propagation.  The  effects  of 
signal  decorrelation  are  then  presented  in  a  tissue  mimicking  phantom  data  set.  The 
chapter  ends  with  experimental  results  from  2D  phantom  and  2D  ex-vivo  prostate 
data. 

Chapter  3  elaborates  on  the  3D  extension  of  the  robust  method  for  displacement 
estimation.  The  methods  section  contains  a  discussion  about  the  benfits  of  using 
random  seed  RF  posts  throughout  the  3D  volume  versus  estimating  displacement 
frame-by-frame.  Experimental  results  are  presented  from  tissue  mimicking  phantom 
and  from  in-vivo  porcine  hepatic  tissue  following  thermal  ablation. 

In  Chapter  4  we  present  a  robust  algorithm  for  ultrasound  elastography  mosaicking. 
Methods  and  experimental  results  are  presented  for  2D  pair-wise,  2D  multi-image  and 
3D  multi- volume  mosaicking.  Tissue  mimicking  phantoms  are  used  in  the  experimen¬ 
tal  evaluation. 

Displacement  estimation  methods  for  thermal  imaging  are  presented  in  Chapter 
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5.  Using  an  iterative  motion  compensation  approach,  we  detect  and  subtract  the 
extraneous,  non  heat-induced,  tissue  motion.  The  efficacy  of  the  method  is  evaluated 
on  ablated  lesions  using  ex-vivo  chicken  breast  tissue.  We  also  present  a  statistical 
analysis  on  the  ability  of  the  method  to  predict  cooler  versus  warmer  regions  of  the 
tissue. 

Finally,  in  Chapter  6,  we  summarize  the  conclusions  of  our  research  work  and  future 
directions. 
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Chapter  2 

Robust  Displacement  Estimation 
for  Ultrasound  Elastography 

2.1  Introduction 

Ultrasound  elasticity  imaging  is  easy  to  use,  cheap  and  portable,  as  it  requires  no  extra 
hardware.  This  makes  it  particularly  appealing  for  medical  applications;  diagnosis  or 
monitoring  could  be  done  at  the  patient  bed  side.  Despite  much  research  effort  and 
some  reported  success  [11,15,16,58],  elastography  is  still  not  routinely  used  in  clinical 
practice.  The  main  reason  is  the  difficulty  in  elastogram  interpretation,  as  images 
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can  be  highly  qualitative  and  hard  to  reproduce  reliably  by  different  operators.  In 
quasistatic  elasticity  imaging  the  tissue  is  compressed  and  data  is  aquired  before  and 
after  the  deformation.  Elasticity  algorithms  usually  involve  two  steps:  1)  the  pre- 
and  post-compression  ultrasound  raw  frequency  (RF)  data  is  compared  to  estimate  a 
displacement  map,  and  then  2)  a  strain  image  is  obtained  by  axial  differentiation  of 
the  displacement  data.  The  speed  and  accuracy  of  elasticity  algorithms  are  influenced 
most  by  the  displacement  estimation  step.  The  user  is  required  to  apply  very  careful 
tissue  compression  in  the  axial  direction  in  order  to  avoid  signal  decorrelation  due  to 
probe  slippage  and  out-of-plain  motion  between  the  pre-  and  post-compression  RF 
frames.  Even  with  a  skilled  operator,  other  sources  of  decorrelation  can  also  affect 
the  estimation  of  tissue  displacement,  such  as  movement  of  structures  as  the  patient 
breathes  or  incoherent  fluid  motion  inside  blood  vessels.  The  resulting  strain  images 
present  with  motion  artifacts,  which  could  be  easily  misinterpreted  as  lesions  (Figure 
2.1  b,d).  The  goal  of  achieving  reliable,  repeatable,  user  independent  elastograms 
depends  heavily  on  solving  the  decorrelation  problem. 

Among  the  approaches  to  reduce  signal  decorrelation  effects,  Pesavento  et  al.  [26]  and 
Zhu  et  al.  [27]  were  the  first  to  propose  taking  advantage  of  displacement  continuity 
within  the  tissue.  In  their  block  matching  algorithms,  each  point’s  displacement  is  ini¬ 
tialized  to  the  previously  calculated  displacement  value  of  it’s  neighbor.  Other  similar 
methods  tracked  displacement  form  row  to  adjacent  row  [28],  column-to-column  [29], 
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Integer  Displacement  Profile  for  Seed  RF-lines  249  -  255 


Displacement 


(a) 


(b)  (c)  (d) 

Figure  2.1:  Integer  DP  displacement  estimation  for  seed  RF  lines  249  -  255  (a).  Note 
the  areas  (for  lines  250  and  252  respectively)  which  exhibit  a  change  in  displacement 
slope.  Strain  images  with  corresponding  artifacts  for  seed  lines  250  (b),  251  (c),  252 

(d) 
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and  diagonally  [31].  Jiang  el  al  [30]  and  Rivaz  et  al.  [34]  posed  the  displacement 
estimation  as  an  optimization  problem  solved  by  dynamic  programming  (DP).  Rivaz 
et  al  [35]  further  regularized  the  displacement  estimation  by  introducing  a  2D  analitic 
minimization  (AM)  technique,  which  greatly  improved  the  speed  of  the  displacement 
estimation  without  sacrificing  accuracy.  The  speed-up  comes  from  reducing  the  costly 
DP  computation  to  only  one  RF  line  pair,  known  as  the  seed  line.  For  the  rest  of  the 
RF  line  pairs,  the  displacement  is  estimated  by  propagating  the  seed  values  laterally 
throughout  the  entire  image. 

The  contribution  of  this  chapter  is  the  development  of  robust  methods  and  quality 
metrics  which  improve  on  the  2D  AM  technique  [35].  The  algorithm  addresses  the 
choice  of  one  or  more  robust  starting  seed  RF  lines,  as  well  as  the  propagation  of 
displacement  values  from  column  to  column.  Compared  to  the  original  algorithm, 
the  current  implementation  is  more  robust  to  signal  decorrelation  and  it  can  estimate 
tissue  motion  even  in  poorly  correlated  regions  of  the  ultrasound  images. 

Chen  et  al.  [33]  implemented  an  algorithm  governed  by  the  presence  of  high-quality 
data  areas,  where  displacement  is  propagated  around  these  regions  first  and  poorly 
correlated  data  zones  are  estimated  last  to  limit  their  overall  influence.  They  did  not 
consider  a  preferential  displacement  propagation  direction  and  they  also  rejected  the 
displacement  continuity  assumption.  This  resulted  in  some  areas  not  being  displayed 
to  the  user,  as  the  display  of  strain  was  suppressed  in  low  quality  regions.  Our  work’s 
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additional  contribution  is  that  it  addresses  the  continuity  problem  by  using  data 
quality  metrics  which  can  detect  changes  in  the  degree  of  correlation,  allowing  for 
reliable  estimation  of  displacement  even  in  poorly  correlated  regions. 

The  remainder  of  this  chapter  is  organized  as  follows.  Section  2  summarizes  the  DP 
technique  for  tissue  displacement  estimation  and  the  2D  AM  regularization.  The 
robust  methods  for  seed  RF  line  selection  and  for  correlation-guided  displacement 
propagation  are  discussed  in  Section  3,  followed  by  validation  experiments  using  tis¬ 
sue  mimicking  phantoms  and  ex-vivo  human  prostate  data  in  Section  4.  Section  5 
concludes  the  chapter  and  future  research  directions  are  discussed. 


2.2  Prior  Work 


2.2.1  2D  Dynamic  Programming  Displacement  Es¬ 
timation 

We  first  summarize  the  DP  technique  for  2D  displacement  estimation  [34],  Consider 
two  ultrasound  RF  frames  I\(i)  and  I2(i)  acquired  before  and  after  tissue  compression. 
Let  n  be  the  number  of  RF  lines,  with  each  signal  sampled  at  i  —  1,  2  •  •  •  m.  Rivaz 
estimates  the  integer  axial  a*  and  lateral  l,  displacements  of  each  RF  line  pair  using 
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DP  [?].  The  cost  function  is  generated  combining  the  prior  of  displacement  continuity 
(smoothness  term)  and  an  amplitude  similarity  term.  The  smoothness  term  takes  into 
account  displacement  continuity  in  both  axial  and  lateral  direction: 

Rj(a-i,  lii  &i— I?  li—  l)  ^a(*T  O— l)  +  &l(li  ^i— l)  (2.1) 

where  aa  and  cq  are  axial  and  lateral  regularization  weights  respectively. 

The  distance  between  pre-  and  post-compression  signals  for  line  pair  j  can  be  written 
as: 

A(i,  j,  cti,  li)  =  [Ii(i,j)  —  12(1  +  auj  +  li )]2  (2-2) 


The  cost  function  at  the  ith  sample  of  the  jth  A-line  becomes: 


Cj(ai,li,i)  =  mmda4l 


{ 


Cj{daA,i  — j  (da  ,2) 
2 


(2.3) 


where  w  is  a  regularization  weight  for  smoothness;  da  and  di  are  temporary  axial 
and  lateral  displacements  which  are  varied  in  order  to  minimize  eqn.(2.3).  The  inte¬ 
ger  displacement  estimates  are  then  refined  to  subpixel  displacement  estimation,  by 
comparing  the  original  pre-compression  signal  I\  (not  downsampled)  with  the  post¬ 
compression  signal  I2  upsampled  by  a  factor  of  7  using  parabolic  interpolation  [34], 
Repeating  the  refinement  procedure  n  times  results  in  a  refinement  factor  of  (1)"  [34], 
The  process  is  repeated  for  all  n  RF  line  pairs,  which  makes  the  entire  displacement 
estimation  algorithm  very  computationally  expensive. 
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2.2.2  2D  Analitic  Minimization  Displacement  Es¬ 
timation 

In  the  Analitic  Minimization  (AM)  method,  the  speed  of  the  algorithm  is  improved 
without  sacrificing  accuracy.  Axial  a*  and  lateral  lL  integer  displacements  are  first 
obtained  using  DP  on  only  one  RF  line  pair,  called  the  seed  line.  The  subsample 
displacement  estimation  algorithm  continues  as  follows: 

1.  a*  and  U  are  first  subsampled  using  linear  interpolation.  Then  they  become 
initial  guesses  for  the  2D  AM  subsample  displacement  estimation  for  the  seed 
line  [35]. 

Let  s  be  the  seed  RF  line.  The  aim  is  to  calculate  A  a,  and  A/*  such  that  the 
duple  (oj  +  Aa(,  l,  +  A/j)  gives  the  axial  and  lateral  displacements  at  the  sample 
i.  The  regularized  cost  function  becomes  [35]: 

Cs(Aui, ...,  Au^j,  A  L, ...,  A  Ini) 

=  ZL  1{[-^1(b  s )  —  I2  (i  +  +  A  a*,  s  +  li  +  A  li)]2  +  (2-4) 

+a  (a*  +  Aa*  —  aj_i  —  Aaj_i)2  +  (3a  (li  +  A/j  —  i  —  AZj_i)2} 

, where  a  and  f3a  are  regularization  terms  which  ensure  continuity  in  displace¬ 
ments  with  respect  to  the  top:  a  for  axial  and  fta  for  lateral. 

2.  Propagate  the  solution  of  the  seed  RF  line  from  column  to  column  to  the  left 
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and  right  of  the  frame,  using  the  displacement  of  the  previous  RF  line  as  an 
initial  estimate.  The  regularized  cost  function  for  the  jth  RF  line  becomes  [35] : 

Cj  ( Aci^ , .  •  • ,  Aum,  Al^ , . . . ,  Alm) 

=  EL  1{[Ii(hj)  ~  Ah'  +  +  A.a,i,j  +  k  +  A/*)]2  + 

To;  (a,  +  Aaj  —  a*_i  —  Aa,_i)  +  f3a  (Zj  +  AZj  —  Z*_i  —  AZj_i)  + 

I  S,  (/,-  +  Ali  —  Zjj-l)2} 

, where  hj-i  is  the  lateral  displacement  of  the  previous  RF  line,  a,  f3a  and  0t 
are  regularization  terms  which  ensure  continuity  in  displacements  with  respect 
to  the  top  (axial  a),  and  the  top  and  left/right  (lateral  /3a  and  0). 

The  presence  of  decorrelation  affects  the  displacement  estimation  algorithm  in  two 
ways.  Choosing  the  seed  RF  line  in  a  decorrelated  region  will  result  in  a  poor  dis¬ 
placement  estimate  for  that  line,  as  well  as  allow  incorrect  estimates  to  propagate 
laterally  though  the  AM  part  of  the  method.  The  choice  of  a  robust  seed  RF  line 

does  not  guarantee  proper  estimates  for  the  entire  frame.  If  the  AM  displacement 

propagation  encounters  a  wide  region  of  poorly  correlated  RF  data,  it  is  unlikey  the 
estimates  will  mentain  correct  values  as  the  region  is  traversed,  potentially  corrupting 
even  the  estimates  in  the  highly  correlated  region  beyond.  Methods  are  needed  both 
for  robust  seed  line  selection  and  also  for  displacement  propagation. 


(2.5) 
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2.3  Methods 

We  are  porposing  a  set  of  methods  to  address  the  presence  of  signal  decorrelation.  The 
first  subsection  2.3.1  presents  two  tools  which  will  constrict  the  choice  of  seed  line  in  an 
area  unaffected  by  decorrelation  to  insure  a  robust  starting  point  for  the  displacement 
estimation  algortihm.  The  next  subsection  2.3.2  introduces  quality  metrics  to  be 
used  in  displacement  propagation,  while  subsection  2.3.3  makes  the  case  for  dividing 
the  image  frames  in  multiple  sub-images,  each  with  their  own  seed  line,  to  allow 
parallelization  and  speed  up  of  the  overall  run  time  without  sacrificing  accuracy. 


2.3.1  Seed  RF-line  selection 

The  following  methods  address  the  selection  of  a  robust  seed  RF  line,  unaffected  by 
signal  decorrelation. 

•  Displacement  Slope 

In  a  continuous  piece  of  tissue,  the  axial  displacement  profile  resulting  from  a 
stress  field  induced  by  applied  compression  has  a  monotonous  ramp  [59].  A 
pair  of  decorrelated  RF  signals  could  affect  the  DP  displacement  estimation 
algorithm.  Instead  of  a  globally  optimal  solution,  the  estimate  could  exhibit 
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Figure  2.2:  Displacement  slope.  The  3  (three)  positions  where  the  change  in  slope  is 
exhibited  could  be  consecutive  (a)  or  not  (b) 

regions  of  locally  optimal  solutions.  We  hypothesized  that  for  small  deforma¬ 
tions,  these  locally  optimal  solutions  would  correspond  to  perturbations  in  the 
monotonously  decreasing  slope  of  their  displacement  profile,  which  in  turn  would 
result  in  artifacts  in  the  final  strain  image  (Fig.  2.1).  A  change  in  the  slope 
along  the  displacement  profile  for  at  least  e  consecutive  data  points  indicates  a 
region  of  poorly  correlated  data.  In  our  phantom  study,  we  found  that  a  value 
of  e  =  3  was  sufficient  to  detect  artifact-inducing  regions  of  signal  decorrelation, 
as  explained  in  2.4.  Note  that  the  three  data  points  don’t  need  to  be  consecu¬ 
tive,  as  long  as  the  ramp  stays  flat  or  continues  to  change  direction  (Figure  2.2 
a,  b). 

•  Displacement  Stability 

In  the  DP  algorithm  [34],  w  is  a  smoothness  regularization  parameter  (2.3);  it 
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prevents  regions  with  high  local  decorrelation  from  introducing  errors  in  dis¬ 
placement  estimation,  but  if  chosen  too  large  would  result  in  oversmoothing. 
We  hypothesized  highly  correlated  pairs  of  RF  lines  would  give  consistent  dis¬ 
placement  estimation  results  regardless  of  the  chosen  value  for  w.  Regions  of 
poorly  correlated  RF  data  would  give  different  displacement  estimation  results, 
given  different  values  of  w  (in  a  chosen  range).  We  thus  compute  the  metric  I 
for  instability  as: 

/  =  the  percentage  of  data  points  along  an  RF  line,  which  exhibit  a  discrepancy 
in  the  displacement  estimate  of  at  least  S  units  (pixels). 

A  high  value  of  I  suggests  a  poorly  correlated  RF  signal  pair.  In  our  phantom 
study,  we  found  that  a  value  of  5  =  3  was  sufficient  for  the  detection  of  signal 
decorrelation,  as  explained  in  2.4. 


2.3.2  Robust,  High-Quality  Displacement  Propa¬ 
gation 

At  the  end  of  the  displacement  estimation  process  on  the  jth  pair  of  RF  lines,  A,  = 
a,  +  A  a*,  Li  =  li  +  Alj  are  the  axial  and  lateral  displacements  at  each  sample  i  (eq. 
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2.5).  Using  the  estimated  displacement  values,  and  the  raw  RF  signals  pre-  and  post¬ 
compression,  we  can  calculate  how  well  the  estimates  fit  the  data.  A  high-quality 
estimate  would  exhibit  a  high  degree  of  correlation  between  the  RF  signals.  We  chose 
the  magnitude  of  the  normalized  cross-correlation  as  an  indicator  for  the  degree  of 
matching  between  the  pre-  and  post-compression  data.  For  the  jth  pair  of  RF  lines, 
we  computed: 


c(hV),hU))  = 


Em1Rl(iJ)-/l)(/2(*+A4J+Li)-/2) 


VEirViOT-h)2  £U1p2b+T,j+£i)-f2)2 

where  I\  and  I2  are  the  means  of  RF  values  along  the  jth  line. 


(2.6) 


C(h  (j),  I2 (j ) )  is  a  good  indicator  of  the  quality  of  the  DP  +  AM  displacement  esti¬ 
mate  for  line  j.  As  the  array  of  A^s  and  L,s  become  the  initial  guesses  for  propagating 
forward  to  line  (j  +  l)th  (eq.  2.5),  we  can  assess  if  the  displacement  propagation  should 
continue  or  not.  A  poor  initial  guess  would  result  in  a  potentially  corrupted  forward 
estimation.  Even  starting  with  a  high-quality  estimate  could  result  in  a  corrupt  for¬ 
ward  estimation  if  the  adjacent  RF  line  pair  ( (j  +  1))  exhibits  a  region  of  poorly 
correlated  data.  We  can  compute  the  same  correlation  metric  (eq.  2.6)  using  the 
estimated  displacements  for  line  j  with  the  raw  RF  data  for  line  j  +  1: 


Cfwd(h(j  +  1),  h(j  +  1)) 


_ Eh  1(-^i(^ii+l)~^i)(^2(t+Ai,(j+l)+Lj)— 12) _ 

A/EUViTi+d-fiPEiT 1(72h+Ai,(i+i)+Li)-/2)2 


(2.7) 
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where  A,  and  L,  are  the  displacements  for  RF  line  j,  and  I\  and  /2  are  the  means  of 
RF  values  along  the  (j  +  l)th  line. 

As  discussed  in  section  2.1,  tracking  algorithms  which  use  the  quality  of  data  as 
an  indicator  of  signal  decorrelation,  fail  to  produce  displacements  estimates  in  the 
presence  of  high  noise  or  signal  decorrelation  [33].  As  suggested  by  [33],  a  quality 
metric  enforcing  continuity  of  displacement  estimates  would  be  desirable,  to  avoid 
differentiation  across  displacement  discontinuities  which  can  generate  artifacts.  To 
ensure  a  continuity  of  high  quality,  highly  correlated  data,  we  compute  the  ratio  p  as 
follows: 


P 


Cfwd  —  C 

c 


(2.8) 


2.3.3  Robust  2D  Elastography  Method  with  Mul¬ 
tiple  Seed  Lines 

Given  two  ultrasound  RF  frames  I\  and  /2  which  exhibit  areas  of  poor  correlation, 
we  are  proposing  a  displacement  estimation  method  which  relies  on  the  selection  of 
a  robust  starting  seed  RF  line,  followed  by  displacement  propagation  guided  by  each 
adjacent  line’s  underlying  degree  of  similarity. 
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We  consider  the  question  of  selecting  a  single  starting  location,  or  multiple  seed 
lines.  As  displacement  is  tracked  from  column-to-column,  the  presence  of  a  poorly 
correlated  signal  pair  would  stop  the  propagation,  practically  requiring  the  selection 
of  a  new  seed  RF  line,  on  the  other  side  of  the  troublesome  area.  To  reduce  the  time 
of  the  displacement  estimation  over  the  entire  image,  one  could  divide  the  image  into 
K  sub-regions  from  the  start,  each  requiring  a  robust  seed  RF  line.  The  selection  of 
multiple  robust  seed  RF  lines  presents  the  advantage  of  initializing  in  multiple  high 
correlation  regions,  incresing  the  chances  for  a  more  robust,  artifact-free  elastogram. 

Let  each  sub-image  /*,  (k  =  1..K )  contain  m/,:  RF  lines.  We  first  select  a  seed  fine  as 
follows  (Figure  2.3): 

1.  Select  p  —  5  random  potential  candidates  for  the  seed  RF  line  pair. 

2.  Compute  integer  displacement  estimation  p  =  5  times  for  each  pair,  using  p 
random  w  values  in  a  chosen  range  (see  section  2.4.1). 

3.  Keep  only  the  pairs  which  satisfy  the  monotonous  displacement  slope  criteria 
for  ALL  values  of  w  (2.3.1). 

4.  From  the  remaining  pairs,  the  one  with  the  lowest  instability  value  I  becomes 
the  seed  RF  line  (2.3.1). 

After  the  selection  of  a  robust  seed  RF  line  j,  the  estimated  displacement  values  are 
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Seed,  Selection 


i  i 

p  =  5  seed  candidates  p  =  5  w  values 

1 — i — 1 

DP  integer  displacement 


Is 

displacement  slope 
monotonous  ? 


|Yes 

Select  seed  with  smallest 
instability  value 


No 


repeat 


Lk+l 


Yes 


No 


Displacement  Propagation 

AM  sub-integer  displacement 

l 

Is  C  >  T\  ?  - 

YesJ 

IsC^Ti? 

Yes  J 

isp=i^i>t2? 


No 


No 


Next  RP  line  pair 


Figure  2.3:  Algorithm  flowchart.  Left  side  represents  the  seed  selection  step,  while 
right  side  presents  the  displacement  propagation  decisions.  If  the  thresholds  are  not 
met,  propagation  stops  and  the  remaining  portion  of  the  RF  frame  will  be  treated  as 
a  new  sub- image  h+i,  where  a  new  seed  RF  line  will  need  to  be  selected. 
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propagated  lateraly  to  the  adjacent  lines  until  all  r/p.  lines  are  processed.  For  each 


adjacent  line  pair  j  +  1,  using  as  initial  displacement  estimate  the  values  calculated 

I  cIwd-c  I 


for  pair  j,  we  calculate  C,  Cfwd,  and  p  = 


c 


as  indicators  of  data  quality 


(2.3.2).  Two  thresholds  govern  the  following  set  of  conditions  which  will  guide  the 
displacement  propagation: 


1.  If  the  jth’s  displacement  estimate  is  a  good  fit  for  line  pair  j,  C  should  be  bigger 
than  T\. 

2.  If  the  jth,s  displacement  estimate  is  a  good  initial  guess  for  line  pair  j  + 1,  Cfwd 
should  be  bigger  than  T\. 

3.  To  ensure  a  continuity  of  high  quality,  highly  correlated  data,  p  should  be 
smaller  than  r2. 


We  study  the  impact  of  w  on  the  instability  value  /,  as  well  as  the  choice  of  thresholds 
Ti  and  t2  in  section  2.4.1.  If  all  the  conditions  are  met,  the  displacement  for  line  j  +  1 
will  be  estimated  according  to  equation  2.5,  and  then  the  same  set  of  conditions  will 
be  used  to  asses  the  quality  of  the  next  RF  line  pair  j  +  2.  If  the  thresholds  are  not 
met,  propagation  stops  and  the  remaining  portion  of  the  RF  frame  will  be  treated  as 
a  new  sub-image  h+u  where  a  new  seed  RF  line  will  need  to  be  selected  (Figure  2.3). 
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2.4  Results  and  Discussion 

2.4.1  Displacement  estimation  in  2D  phantom  data 

To  study  the  choice  of  algorithm  parameters  and  their  impact,  we  palpated  and 
scanned  a  CIRS  (CIRS  Inc.  Norfolk,  Virginia)  breast  elastography  phantom  model 
059  (Figure  2.4).  According  to  the  manufacturer  specifications,  the  phantom  pre¬ 
sented  with  a  10  mm  diameter  dense  mass  at  least  two  times  stiffer  than  the  back¬ 
ground,  which  had  an  elastic  modulus  of  20 kPa  ±  5 kPa.  The  raw  ultrasound  data 
were  obtained  from  ACUSON  Antares  (Siemens  Medical  Solutions  USA,  Malvern, 
PA,  USA),  while  the  tissue  was  palpated  freehand  using  an  ultrasound  transducer 
(VF13-5SP)  at  a  center  frequency  of  7.27  MHz.  The  Axius  Direct  Ultrasound  Re¬ 
search  Interface  was  used  to  enable  RF  acquisition  at  a  sampling  rate  of  40MHz,  with 
each  frame  comprising  508  lines  of  1700  samples. 

Two  RF  frames  were  selected  which  exhibited  some  unknown  regions  of  poorly  cor¬ 
related  RF  data.  In  order  to  evaluate  the  magnitude  of  the  problem,  we  first  set  out 
to  evaluate  the  percentage  of  decorrelated  RF  line  pairs.  Integer  displacements  were 
estimated  for  each  RF  line  pair  using  DP  (equation  2.3).  Every  line  pair  was  treated 
as  a  seed  and  thus  processed  individually,  using  no  prior  information  for  an  initial 
guess.  Visualizing  the  individual  axial  displacements  as  a  unified  map  makes  it  easy 
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(a)  (b) 

Figure  2.4:  CIRS  breast  phantom  059.  The  elasticity  of  each  dense  mass  is  at  least 
two  times  greater  than  the  elasticity  of  the  background,  which  has  an  elastic  modulus 
of  20kPa  ±  5  kPa.  They  range  in  size  between  3  and  10  mm  (b) 

to  observe  potential  discontinuities  and  poorly  correlated  areas  (Figure  2.5). 

Further  more,  the  choice  of  w  value  affects  the  final  strain  image  (Figure  2.6).  In  the 
DP  displacement  computation,  w  is  a  smoothness  regularization  parameter  (equation 
2.3)  intended  to  prevent  poorly  correlated  regions  from  introducing  errors  in  displace¬ 
ment  estimation.  5346  strain  images  were  obtained  with  the  2D  AM  method  using 
each  RF  line  as  a  seed,  each  for  eleven  values  for  w  between  0.10  and  0.60  (Table 
2.1).  Knowing  the  shape  and  size  of  the  expected  lesion,  we  visually  inspected  each 
resulting  elastogram  for  the  presence  of  artifacts.  Some  lines  produced  very  faint, 
very  small  artifacts  on  the  order  of  a  couple  of  pixels  which  were  not  propagated  to 
the  adjacent  lines;  if  these  artifacts  were  not  easily  identifiable  at  first  inspection,  the 
corresponding  line  was  categorized  as  indeterminate  (Table  2.1). 
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Figure  2.5:  Breast  phantom  data.  Integer  displacement  estimation  using  dynamic 
programming  method.  Note:  RF  lines  pairs  #13  -  498  are  each  used  as  seed.  Many 
areas  prone  to  produce  artifacts  are  relatively  easy  to  identify  (arrows). 
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Figure  2.6:  Examples  of  artifacts  in  strain  images  for  various  values  of  w.  First  row 
represents  strain  images  originating  from  seed  RF  line  #487,  second  row  from  seed 
RF  line  #55.  Seed  line  is  emphasized  in  color  yellow. 
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Table  2.1:  Percentage  distribution  for  seed  RF  lines,  given  multiple  values  for  w 
(smoothness  regularization  parameter) 


Lines  (%) 

0.10 

0.15 

0.20 

0.25 

0.30 

w 

0.35 

0.40 

0.45 

0.50 

0.55 

0.60 

W/o  artifacts 

44.2 

57.8 

61.3 

68.1 

65.0 

61.7 

53.3 

53.1 

43.4 

32.1 

24.3 

W/  artifacts 

55.8 

42.2 

33.6 

30.0 

30.5 

32.0 

41.8 

46.9 

56.6 

67.9 

75.8 

Indeterminate 

- 

- 

5.1 

1.9 

4.5 

5.3 

4.9 

- 

- 

- 

- 

For  the  breast  phantom  data,  we  focused  our  remaining  analysis  on  values  of  w  in 
the  0.20  -  0.35  range,  where  over  60%  of  lines  produced  no  strain  artifacts  (Table 
2.1).  For  this  w  range,  113  (23.25  %)  lines  produced  artifacts  for  all  four  values 
of  w  (0.2,  0.25,  0.3,  0.35),  263  (54.12%)  lines  never  produced  artifacts  and  110 
(22.63%)  lines  oscillated  in  their  behavoir. 

The  monotonous  displacement  slope  criterium  (2.3.1)  was  very  effective  as  an  initial 
test  for  filtering  out  most  of  the  poorly  correlated  line  pairs.  As  a  filter  by  itself,  it 
had  a  74.18%  sensivity  and  77.78%  specificity. 

The  instablity  metric  I  (2.3.1)  clearly  differentiated  between  high  quality  data  and 
regions  of  signal  decorrelation  (Figure  2.7).  Our  algorithm  uses  this  metric  to  select 
from  the  seed  candidates  the  line  pair  with  the  lowest  I  value.  Our  study  indicates 
one  could  also  constrain  /  to  be  within  a  user-selected  threshold  of  high  instability; 
boxplot  in  figure  2.7  suggests  a  possible  value  could  be  chosen  in  the  3-5%  range. 
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Average  instability  over  w  =  {0.20,  0.25,  0.30,  0.35} 


Figure  2.7:  Box  and  wiskers  plot:  Instability  metric  I  (  averaged  over  w  =  0.2,  0.25, 
0.3,  0.35).  Values  corresponding  to  the  artifact  producing  lines  are  shown  in  blue 
while  the  robust  lines  are  shown  in  red. 

Using  the  phantom  data,  we  evaluated  the  magnitude  of  the  normalized  cross-correlation 
C{I\  (j).  (equation  2.6)  as  an  indicator  of  the  degree  of  matching  between  the 

pre-  and  post-compression  data.  The  spread  and  distribution  of  C{Ii (j),  V2 (j ) )  with 
respect  to  w  is  presented  in  Figure  2.8.  Once  again,  w  values  in  the  0.20  -  0.35 
range  resulted  in  a  clear  differentiation  of  robust  vs.  artifact  producing  lines.  The 
results  in  Figure  2.8  determined  our  choice  of  values  for  the  two  thresholds  used  in 
the  correlation- guided  displacement  propagation:  we  reject  all  lines  with  C  or  Cfwd 


40 


CHAPTER  2.  ROBUST  DP 


NCC  of  sub-integer  displacement  values  for  each  RF  line  pair 


less  than  T\  =  75%  or  lines  for  which  the  ratio  p 


C fwd  C 

c 


is  higher  than  72 


10%. 


Figure  2.8:  Box  and  wiskers  plot:  magnitude  of  normalized  cross-correlation  C.  Val¬ 
ues  corresponding  to  the  artifact  producing  lines  are  shown  in  blue  while  the  robust 
lines  are  shown  in  red. 


2.4.2  Displacement  Estimation  in  2D  human  prostate 
ex  vivo  data 

Ex  vivo  ultrasound  data  was  collected  from  resected  human  prostates,  following 
prostatectomy.  In  accordance  with  the  Institutional  Review  Board  approval,  each 
prostate  specimen  also  underwent  a  post-operative  ultra  high-resolution  MRI  at  9.4 
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Tesla.  The  RF  ultrasound  data  were  obtained  from  ACUSON  Antares  (Siemens 
Medical  Solutions  USA,  Malvern,  PA,  USA),  while  the  specimen  was  palpated  free¬ 
hand  using  an  ultrasound  transducer  (VF10-5)  at  a  center  frequency  of  6.67  MHz. 
The  Axius  Direct  Ultrasound  Research  Interface  was  used  to  enable  RF  acquisition 
at  a  sampling  rate  of  30MHz,  with  each  frame  comprising  350  lines  of  2236  samples. 
Post-operative  MR  scans  were  used  for  anatomical  correlation  with  the  strain  images. 

The  full  results  of  the  study  are  presented  in  Appendix  A.  Here  we  will  present  data 
from  one  specimen,  for  experimental  validation  of  our  algorithm  on  ex  vivo  data. 

The  robust  multi-seed  algorithm  was  used  for  displacement  estimation,  w  ranged 
from  0.20  to  0.35  for  the  seed  selection  step,  and  T\  =  75%  and  r2  =  10%  for  the 
displacement  propagation  step.  We  discuss  in  details  below  how  we  dealt  with  two 
challenging  situations. 

Given  sub-image  Ik,  one  or  more  RF  line  pairs  could  be  highly  decorrelated,  pre¬ 
venting  displacement  propagation  and  effectively  stopping  the  progression  of  the  al¬ 
gorithm.  In  this  prostate  data  example,  we  considered  the  initial  sub-image  as  the 
entire  frame.  RF  line  pair  #146  was  selected  as  seed  (Figure  2.9  a,  dotted  line).  Dur¬ 
ing  displacement  propagation,  the  algorithm  encountered  two  decorrelated  regions 
indicated  by  values  for  C  under  the  threshold  t\  (see  circled  areas  in  Figure  2.9  a). 
The  values  for  Cfwd  closely  follwed  C  and  also  fell  under  the  threshold  (Figure  2.9  b, 
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c).  Two  new  sub-images  were  defined  (I\  to  the  left  and  /2  to  the  right)  for  which 
the  algorithm  started  anew  with  the  selection  of  a  robust  seed  line. 

The  second  situation  arises  when  in  a  remaining  (small)  sub-image,  the  RF  data 
could  exhibit  such  poor  correlation  that  no  line  would  fit  the  criteria  for  a  robust 
seed  selection.  In  our  implementation,  we  maintain  an  array  of  size  n  containing  the 
values  of  C  for  each  RF  line  pair.  They  are  initialized  to  0  (zero)  and,  as  each  sub¬ 
image  Ik  is  being  processed,  we  update  the  values  of  C  for  each  RF  line  pair.  When 
a  seed  selection  is  not  possible,  some  lines  remain  unprocessed.  After  all  sub-images 
finish  propagation,  a  linear  search  is  used  for  the  detection  of  unprocessed  lines.  For 
each  such  line,  Cfwd  and  p  =  — #4 —  are  calculated  for  both  possible  directions  of 
propagation  (from  the  adjacent  left  line  or  right  line).  The  highest  quality  boundary 
determines  which  displacement  values  will  be  used  as  an  initial  guess.  This  insures 
possible  small  artifacts  will  be  confined  to  small  regions  and  will  not  impact  the 
displacement  estimate  of  other  lines. 

In  our  prostate  data  example,  we  needed  to  select  a  seed  line  in  sub-image  R  defined 
to  the  left  of  the  frame,  consisting  of  thirty-four  RF  line  pairs.  Only  one  line  pair 
passed  the  displacement  slope  criterium.  Continuing  on,  our  threshold  T\  of  75%  was 
not  met,  so  we  were  left  with  a  sub-image  with  no  suitable  seed  line.  The  lines  in 
R  were  processed  at  the  end,  using  displacement  values  propagated  from  line  #35 
towards  line  #1,  and  the  final  strain  image  (incorporating  the  displacement  values 
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Magnitude  of  Normalized  Cross  Correlation 


(b) 

Magnitude  of  Forward  Normalized  Cross  Correlation 


(c) 

Figure  2.9:  Prostate  data  set  example:  correlation  values  for  each  line  pair  (a), 
forward  correlation  for  displacement  propagation  in  R  (b),  and  R  (c).  Seed  line  is 
shown  by  dotted  line  in  (a).  Also  shown  are  breaking  points  due  to  low  correlation 
values  (circles) 
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Anal  Strain 


(c) 

Figure  2.10:  Prostate  data  set  example:  (a)  an  unsuitable  seed  line  is  selected  for 
sub- image  I\ ,  (b)  no  suitable  seed  line  is  found  in  sub-image  I\,  which  forces  the 
algorithm  to  propagate  displacement  from  line  #35,  and  (c)  9.4  Tesla  MRI  corronal 
scan  of  the  prostate  specimen  showing  hard  lesions  (arrows),  soft  lesions  (dashed 
arrows)  and  urethra  (bracket). 


45 


CHAPTER  2.  ROBUST  DP 


from  sub-image  /2)  is  presented  in  Figure  2.10  b). 

If  T\  is  set  at  a  lower  value  (for  example  70%  instead  of  75%),  one  seed  line  will  be 
accepted  and  displacement  estimates  will  be  propagated  throughout  sub-image  I\. 
The  resulting  elastogram  in  Figure  2.10  a)  looks  very  different  from  the  one  in  b), 
clearly  illustrating  the  importance  of  sellecting  a  high  value  for  threshold  T\.  Figure 
2.10  c)  shows  a  9.4  Tesla  MRI  corronal  scan.  The  pathological  examination  showed 
the  lesions  pointed  by  full  arrows  to  be  hard,  while  the  ones  pointed  by  dashed 
arrows  were  soft.  All  the  lesions  were  uncovered  in  a  very  convincing  manner  by  the 
elastography  algorithm.  Also  visible  in  the  MRI  scan  is  the  urethra,  whose  rigid  walls 
make  it  appear  hard  in  the  strain  image. 


2.4.3  Algorithmic  Complexity 

Making  use  of  multi-threaded  parallel  processing  greatly  reduces  the  computational 
time  for  the  entire  displacement  estimation.  The  number  of  sub-images  K  dictates,  in 
part,  the  total  time.  For  each  sub-image  k  =  1..K,  the  computational  time  is  divided 
between  the  DP  displacement  estimation  for  the  seed  line  selection  and  the  subsequent 
propagation  to  the  remaining  m*,  —  1  lines  in  the  region.  Choosing  a  large  K  divides 
the  image  in  many  small  strips,  reducing  the  value  of  m^-  From  a  run-time  point 
of  view,  this  does  not  have  a  great  reducing  effect,  as  the  AM  part  of  the  algorithm 
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which  is  used  for  displacement  propagation  is  an  order  of  magnitude  faster  than  the 
intial  DP  estimation  for  the  seed  line.  A  value  of  K  too  large  increases  the  chance 
of  encountering  a  sub-image  where  no  suitable  seed  line,  thus  extending  the  overall 
run-time.  Choosing  a  small  K  would  increase  the  chances  of  encountering  a  poor 
quality  data  region,  thus  forcing  a  new  DP  estimation  for  seed  selection  and  affecting 
again  the  total  run-time.  In  our  experience,  initial  sub-images  of  approximately  10% 
of  the  total  number  of  lines  produced  the  best  results. 


2.5  Conclusion  and  future  work 

We  have  proposed  a  robust  method  for  displacement  estimation  in  ultrasonic  strain 
imaging.  This  method  enhances  the  dynamic  programming  approach  to  displacement 
estimation,  improving  its  ability  to  overcome  displacement  discontinuities  and  regions 
of  poorly  correlated  RF  data.  The  innovative  methods  proposed  for  seed  line  selec¬ 
tion  focus  on  robustness  as  well  as  stability  over  varying  algorithm  parameters.  The 
method  also  addresses  the  robustness  of  displacementt  propagation  using  analytic 
minimization.  Enforcing  robustness  and  stability  does  not  add  a  significant  overhead 
to  the  total  run-time  of  the  displacement  algorithm.  The  method  is  highly  paral- 
lelizable  at  every  step,  from  the  seed  selection  tools  to  the  displacement  propagation. 
The  new  algorithm  is  one  step  closer  towards  achieving  repeatable,  user- independent 
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elastograms. 
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Chapter  3 

3D  Displacement  Estimation  for 
Ultrasound  Elastography 

3.1  Introduction 

The  introduction  of  ultrasound  elasticity  by  Ophir  in  1991  was  soon  followed  by  three 
dimensional  attempts.  Early  work  relied  on  multiple  frames  being  acquired  using  a 
conventional  2D  ultrasound  transducer  [60,39],  while  more  recently  3D  ultrasound 
transducers  were  employed  [43,32],  Similar  to  2D  elastography,  signal  decorrelation 
is  the  main  factor  affecting  the  quality  of  3D  strain  images.  Additionally,  the  speed 
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Figure  3.1:  Conventional  perpendicular  motion  directions  in  ultrasound  data 

of  displacement  estimation  becomes  even  more  important  for  clinically  significant,  in 
vivo  applications  of  three-dimensional  elasticity  imaging  . 

Two  techniques  have  emerged  for  acquiring  the  raw  signals  used  for  3D  displacement 
estimation.  In  the  first  case,  a  2D  transducer  is  slowly  moved  deliberatly  in  the 
elevational  direction  (Figure  3.1),  while  continuously  acquiring  data  [37,38].  This 
acquisition  mode  results  in  small  variations  in  the  applied  compression  between  ad¬ 
jacent  probe  locations.  This  approach  requires  a  slow  enough  transducer  movement 
to  keep  consecutive  scans  not  too  far  apart  in  the  elevational  direction,  making  the 
method  prone  to  decorrelation  due  to  out-of-plane  motion.  Lindop  et  al  [28]  later 
improved  the  quality  of  strain  volumes  by  applying  a  3D  filter  to  deal  with  this  issue, 
but  the  data  acquisition  step  remained  slow. 
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In  the  second  technique  two  volumes,  pre-  and  post-compression,  are  acquired  sequen¬ 
tially,  and  a  small  pressure  is  applied  to  the  tissue  between  volume  acquisitions  using 
a  mechanical  or  simulated  fixture  [39,40,41,42,43,44],  or  freehand  [32],  This  approach 
requires  great  care  in  applying  even  axial  compression  over  the  entire  tissue  imaged 
to  avoid  signal  decorrelation.  Further  more,  the  operator  has  to  hold  the  probe  very 
still  during  the  lenghty  acquisition  of  the  pre-  and  post-compression  volumes,  which  is 
very  difficult  in  practice  without  the  use  of  steady  mechanical/robotic  arms  or  stages. 

Two  hybrid  acquisition  techniques  stand  out  recently.  Houdsen  et  al.  [45]  used  a 
mechanically-swept  (wobbler)  3D  probe  (Figure  3.2)  to  control  the  elevational  mo¬ 
tion.  While  the  probe  gradually  sweeps  out  a  volume,  it  pauses  at  each  motor  step 
and  collects  several  RF  frames,  as  the  operator  performs  the  tissue  deformation  con¬ 
tinuously,  by  up  and  down  hand  movement  in  the  axial  direction.  This  method  has 
ease  of  acquisition  as  it  bypasses  the  practical  difficulty  of  performing  careful  and  op¬ 
timal  compression  between  pre-  and  post-compression  frames.  It  still  suffers  however 
from  the  speed  of  acquisition,  which  can  take  an  extended  time  for  large  volumes. 

Foroughi  et  al.  used  a  tracked  3D  wobbler  transducer  with  an  attached  position  sensor 
[57].  RF  data  is  continuously  collected  as  the  transducer  sweeps  the  volume,  while 
the  operator  performs  continuous  freehand  compression.  Data  is  acquired  without 
pause  and  the  tracking  information  is  exploited  to  select  suitable  RF  frame  pairs 
with  minimal  out-of-plane  motion.  The  acquisition  of  several  volumes  is  fast  and 


51 


CHAPTER  3.  ROBUST  3D  DP 


3D  Scan 


Motor  sweep 


2D  US  Frames 


3D  US  Volume 


3D  Reconstruction 


Figure  3.2:  3D  wobbler  transducer  principles:  data  acquisition  and  3D  volume 
reconstruction. 

no  operator  training  is  required  as  the  frame  selection  technique  minimizes  signal 
decorrelation  due  to  hand  motion. 

In  this  chapter  we  extend  the  robust  displacement  estimation  to  three  dimenssions. 
The  3D  data  sets  were  collected  using  the  Foroughi  system  [57] .  Next  we  summarize 
the  Foroughi  tracked  ultrasound  elastography  (TrUE)  techinque  [57],  followed  by  a 
description  of  the  three  dimensional  robust  estimation  methodology.  The  chapter 
continues  with  a  section  on  3D  validation  experiments  using  tissue  mimicking  phan¬ 
toms  and  in  vivo  intra-operative  porcine  liver  data  and  concludes  with  a  discussion 
on  future  research  directions. 
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3.2  Tracked  Ultrasound  Elastography  (TrUE) 

For  freehand  3D  data  collection,  the  quality  of  strain  images  becomes  extremely  sen¬ 
sitive  to  the  user’s  level  of  expertise.  Precise  axial  deformation  is  needed  to  avoid 
out-of-plane  motion  which  results  in  signal  decorrelation.  Here  we  give  an  overview 
of  the  Foroughi  system  which  makes  use  of  position  information  to  select  pairs  of 
RF  frames  which  have  a  higher  probability  of  generating  high  quality  strain  [57]. 

A  few  recent  ultrasond  systems  have  already  incorporated  tracking  devices  to  ex¬ 
ploit  position  information  (The  LOGIQ  E9  (GE,  Schenectady,  New  York)  and  the 
SonixGPS  (Ultrasonix  Medical  Corp.,  Richmond,  BC,  Canada)  are  two  examples  of 
such  systems). 

As  data  is  acquired  while  the  2D  element  array  sweeps  the  volume,  synchronized 
position  information  is  recorded  simultaneously.  A  transformation  matrix  is  computed 
which  determines  the  position  and  orientation  of  every  slice.  The  geometry  of  the 
probe  and  the  acquisition  angle  of  the  slice  determine  each  transformation  matrix. 

In  the  case  of  our  transducer,  a  volume  is  constructed  by  sliding  a  linear  array  over 
the  sector  of  a  circle  with  radius  r  as  shown  in  Figure  5.1.  9  is  the  angle  of  the  slice 
at  each  step  of  the  wobbler  motor,  r  and  9  are  known  from  the  probe  manufacturer 
specifications. 

The  position  information  helps  select  pairs  of  RF  frames  which  have  a  high  probability 
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Elevational  (Z) 


Latera 


Figure  3.3:  Overview  of  the  TrUE  approach.  Given  r  and  0,  each  RF  slice  within  a 
volume  is  localized.  Courtesy  of  Foroughi  et  al  [57]. 

of  producing  good-quality  strain  images.  The  goal  is  to  find  frame  pairs  for  which 
the  motion  is  mainly  in  axial  direction,  with  minimal  lateral  and  elevational  motion 
(Figure  3.1).  Foroughi’s  solution  is  found  by  maximizing  the  following  cost  function 


[57]: 


(3.1) 


where  Kx,  Ky,  and  Kz  determine  the  sensitivity  to  motion  in  lateral,  axial,  and 
elevational  directions  respectively,  and  D  =  [DxDyDz]  is  the  sample-wise  root  mean 
square  of  the  displacement.  topt  represents  the  desired  compression,  and  c  is  a  small 
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number  that  limits  the  cost  of  zero  compression. 

Due  to  the  continuous  freehand  deformation,  the  axial  direction  for  each  acquired 
frame  varies  depending  on  the  acquisition  angle  9  (Figure  5.1).  Imperfect  hand  mo¬ 
tion  is  generally  an  obstacle  for  elasticity  algorithms.  Foroughi  et  al  [57]  consider  it 
however  beneficial  for  their  algorithm.  As  they  attempt  to  maximize  the  cost  function 
for  optimal  compression  in  a  given  axial  direction,  they  evaluate  every  frame  at  the 
respective  motor  step  location,  but  also  the  ones  at  one  preceding  and  one  succeeding 
location.  The  result  is  an  improved  strain  quality  at  the  center  slice,  and  also  at  the 
adjacent  slices  [57]. 


3.3  Methods 

3.3.1  Robust  3D  Elastography  with  Multiple  Seed 
Lines 

For  3D  displacement  estimation  we  combine  the  frame  selection  algorithm  of  For¬ 
oughi  et  al  [57]  with  the  robust  methods  presented  in  Chapter  (2).  During  freehand 
palpation  using  a  tracked  ultrasound  transducer,  the  Foroughi  method  automatically 
selects  pairs  of  RF  frames  with  minimal  lateral  and  out-of-plane  motions,  and  also 
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has  the  possibility  to  select  frames  with  a  predefined  compression  range  with  respect 
to  one  another  [57] .  While  the  frame  selection  technique  resolves  signal  decorrelation 
due  to  out-of-plane  motion,  the  robust  2D  displacement  estimation  addresses  small 
local  decorrelations,  which  can  be  present  only  in  a  small  number  of  frames  in  a  swept 
volume.  One  example  is  the  presence  of  fluid  (blood  or  lymphmatic  vessels,  billiary 
ducts)  which  can  affect  the  entire  3D  displacement  estimation. 

The  robust  2D  displacement  estimation  is  performed  after  the  frame  pair  selection. 
The  position  of  each  frame  is  available  to  us  from  the  tracking  information,  so  we 
can  use  it  to  automatically  adjust  the  search  range  for  the  DP  part  of  the  algorithm 
(chapter  2.2.2).  This  results  in  a  faster  integer  displacement  estimation  for  the  selec¬ 
tion  of  seed  lines.  A  strain  image  is  computed  for  each  frame  pair,  then  all  frames  are 
combined  in  one  3D  strain  volume  by  applying  a  scaling  factor  represented  by  a  3D 
smoothing  kernel,  weighted  by  the  mean  strain  of  each  pair.  To  display  the  results, 
strain  images  are  stacked  and  scan-conversion  is  applied  to  the  resulting  volume. 
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3.4  Experimental  Results 

3.4.1  3D  Ex- vivo  Displacement  Estimation 

For  3D  evaluation  we  palpated  and  scanned  a  CIRS  (CIRS  Inc.  Norfolk,  Virginia) 
phantom,  model  049A  (Figure  3.4).  The  ultrasound  data  were  obtained  from  Sonix- 
CEP  ultrasound  system  (Ultrasonix  Medical  Corp.,  Richmond,  BC,  Canada).  The 
phantom  was  palpated  freehand  using  the  4DL14  —  5/38  transducer  at  a  center  fre¬ 
quency  of  6.7  MHz.  A  passive  marker  was  attached  to  the  transducer  and  was  tracked 
using  the  Polaris  optical  tracker  (Northern  Digital  Inc.,  Waterloo,  Ontario,  Canada). 
Customized  in-house  software  was  used  to  enable  continous  acquisition  of  RF  data 
and  synchronized  tracker  readings  through  the  Ultrasonix  Research  Interface.  The 
volumes  consisted  of  120  slices.  About  15  forward  sweep  scans  were  collected  for  each 
series  of  volumes. 

Phantom  model  49a  includes  four  types  of  cylinders  with  varying  diameters,  from  1.58 
to  6.49  mm  (Figure  3.4).  Our  scan  captured  three  of  these  cylinders  with  elasticities 
of  80  kPa,  45  kPa  and  14  kPa,  vs.  25  kPa  for  the  background.  The  elastograms  ob¬ 
tained  using  our  algorithm  captured  both  the  harder  and  the  softer  cylinders  (Figure 
3.5).  The  harder  cylinders  apear  darker  and  the  softer  one  apears  brighter  than  the 
background.  One  of  the  hard  cylinders  is  almost  two  times  harder  than  the  other; 
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(a) 


STEP  TARGET 


(b) 

Figure  3.4:  CIRS  phantom  model  49a.  The  elasticity  of  the  cylinders  is  80  kPa, 
45  kPa,  14  kPa  and  8kPa,  vs.  25  kPa  for  the  background.  The  cylinders  range  in 
diameter  from  1.58  to  6.49  mm  (b). 
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on  the  elastogram  the  left  cylinder  appears  darker  than  the  middle  one  (Figure  3.5  a, 
e).  This  is  important  as  it  underscores  the  algorithm’s  ability  to  differentiate  among 
lesions  with  different  elasticity  values  within  one  large  volume,  using  the  maximum 
range  of  the  wobbler  probe.  In  the  area  scanned,  the  diameters  of  the  cylinders  were 
2.53,  4.05  and  6.49  mm,  which  agrees  with  the  dimensions  on  the  elastograms. 


3.4.2  3D  In  vivo  Displacement  Estimation 

In  the  in  vivo  experiment,  a  pig  was  prepared  to  undergo  liver  ablation  surgery. 
After  gaining  access  to  the  liver,  an  ablation  needle  was  placed  in  a  thick  portion 
of  the  liver  under  ultrasound  guidance.  The  hepatic  tissue  was  ablated  using  the 
RITA  ablation  device  (RITA  Medical  Systems  Inc,  Mountain  View,  California).  The 
resulting  thermal  lesions  were  1  to  2  cm  in  diameter.  The  ablated  zone  was  then 
scanned  following  the  approved  protocol  during  a  breath-hold  period  which  lasted  less 
than  20  seconds  (Figure  3.6(a)).  The  strain  scans  of  the  two  thermal  lesions  apear  in 
Figure  3.6  d-i,  where  the  ablated  regions  are  clearly  visible.  The  first  ablation  appears 
to  be  smaller  than  the  second  one  which  was  confirmed  by  the  gross-pathology  (Figure 
3.6(b),  3.6(c)).  The  diameter  of  the  first  lesion  is  about  1.5  cm  whereas  the  second 
lesion  is  closer  to  2  cm.  The  surgical  gauze  placed  underneath  the  liver  has  created 
some  distortion  in  the  deeper  parts  of  the  scans. 
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Figure  3.5:  Elastograms  of  phantom  model  49a:  (a)  cross-section  of  three  cylinders 
with  different  elasticity  values;  (b-d)  The  axial-lateral  elastograms  of  the  three  cylin¬ 
ders.  (e-g)  Lateral-elevational  elastograms  of  the  three  cylinders.  From  left  to  right, 
the  elasticity  of  the  cylinders  is  80  kPa,  45  kPa  and  14  kPa,  vs.  25  kPa  for  the  back¬ 
ground.  The  harder  cylinders  apear  darker  and  the  softer  one  apears  brighter  than 
the  background. 
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3.5  Conclusion  and  future  work 

As  we  extended  this  method  to  3D  volumes  of  data,  we  considered  a  multi-seed  ap¬ 
proach  where  robust  seed  axial  posts  would  be  selected  at  random  locations  through¬ 
out  the  volume.  In  this  case,  displacement  would  propagate  in  a  radial  fashion,  with 
the  eight  RF  line  pairs  adjacent  to  the  seed  being  evaluated  first.  We  concluded  that 
such  an  approach  did  not  reduce  the  total  run-time,  nor  did  it  appear  to  improve  the 
overall  displacement  estimate.  One  of  the  constraints  of  this  approach  was  the  aspect 
of  propagating  the  displacement  in  the  elevational  direction.  As  the  estimation  gets 
deeper,  the  distance  between  samples  increases  and  continuity  constraints  can  lead 
to  the  wrong  local  solution.  As  capabilities  of  the  3D  wobbler  transducers  improve, 
we  consider  this  area  a  worthy  future  direction. 

Pairing  the  robust  methods  with  the  frame  selection  algorithm  combines  the  strenghts 
of  both  techniques.  Frame  pairs  with  large  out-of-plane  motion  are  avoided,  while 
small  intra-frame  decorrelations  are  addressed  in  a  frame-by-frame  fashion.  In  the 
future,  we  consider  incorporating  a  full  3D  displacement  estimation  technique  to 
enforce  smoothness  constraints  among  the  frame  pairs.  Another  future  direction  is 
the  addition  of  a  real-time  feedback  during  palpation  and  data  collection,  which  would 
signal  when  enough  suitable  frame  pairs  have  been  collected. 
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Figure  3.6:  Ex-vivo  hepatic  ablation,  (a)  3D  transducer  positioned  on  top  of  liver 
during  elastography  scan;  (b)  and  (c)  Cross-sections  of  the  ablated  regions  with  ruler 
visible;  (d-f)  Middle  cross-sections  of  strain  volume  for  ablated  lesion  #1  and  (g-i) 
lesion  #2. 
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Chapter  4 

Ultrasound  Elastography 
Mosaicing 

4.1  Background 

Compared  to  other  imaging  modalities  like  CT  and  MR,  ultrasound  suffers  from  a 
limited  field  of  view.  Monitoring  a  structure  can  be  particularly  challenging  when  it 
is  too  large  to  be  visualized  in  a  single  image  or  3D  sweep.  Size  and  distance 
measurements  are  unreliable  in  large  organs.  Panoramic  imaging  is  emerging  as  a 
prevalent  technique  used  in  widening  the  field  of  view  (FOV)  of  medical  ultrasound 
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images.  Also  referred  to  as  stitching  or  panorama,  the  ultrasound  mosaic  aims  to 
achieve  several  clinical  advantages  which  come  along  with  extended  FOV:  1) 
improving  the  understanding  of  spatial  relationships  among  structures  when  the  size 
of  a  single  image/volume  is  not  large  enough  to  cover  the  entire  area,  2)  allowing  for 
measurements  of  size  and  distance  in  large  organs  and  lesions,  3)  allowing 
multi-modal  registration  and  fusion  with  pre-operative  data  for  guidance  in 
minimally-invasive  interventions.  Since  ultrasound  elastography  was  first  proposed 
two  decades  ago  for  the  clinical  imaging  of  tissue  stiffness,  numerous  clinical 
applications  have  been  investigated,  among  them  imaging  cancerous  tissue  [11], 
ablation  monitoring  [58],  and  the  detection  and  grading  of  deep  vein 
thrombosis  [61].  In  this  work  we  focus  on  extended  FOV  displacement  estimation 
for  quasi-static  ultrasound  elastography.  The  tissue  is  imaged  while  it  is  slowly 
deformed  using  an  external  mechanical  force  and  the  images  are  used  for  the 
estimation  of  tissue  motion  or  displacement  [11]. 

Elastography  maps  the  mechanical  properties  of  tissue  which  can  add  valuable 
features  to  the  B-mode  panorama.  Many  clinical  applications  deal  with  large 
cancerous  lesions  which  expand  beyond  the  span  of  one  ultrasound  image  [15].  An 
ultrasound  elastography  mosaic  can  improve  the  understanding  of  the  size  of  the 
lesion  and  its  layout  among  surrounding  structures.  In  the  ablation  of  hepatic 
cancerous  tissue,  the  size  of  the  HIFU-induced  ablative  lesions  often  exceeds  4  cm  in 
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diameter,  which  is  the  width  of  a  typical  ultrasound  transducer.  Thermal  lesions  are 
not  visible  in  conventional  B-mode  ultrasound  but  a  panoramic  ultrasound 
elastogram  can  help  visualize  the  entire  extent  of  the  ablation  [58],  monitoring  and 
insuring  all  cancerous  tissue  is  ablated.  In  the  assessment  of  venous  thrombi,  a 
combination  of  ultrasound  B-mode  and  Doppler  imaging  help  detect  the  presence  of 
the  blood  clot,  but  it  is  elastography  which  can  provide  its  age  and  clinical 
grading  [61].  An  in-plane  ultrasound  elastography  mosaic  can  provide  mapping  of 
the  thrombi  all  along  the  femoral  vein.  It  could  take  up  to  eight  mosaiced  volumes 
to  depict  an  entire  organ  like  the  liver  or  kidney  [62],  but  having  the  corresponding 
elastography  mosaic  would  allow  for  registration  with  pre-operative  imaging  data 
(CT  or  MRI)  which  would  help  with  intra-operative  navigation.  Although  the 
basics  of  medical  elastography  have  long  been  defined,  new  clinical  applications  are 
constantly  emerging  are  we  are  seeing  an  increasing  commercial  and  clinical  interest. 

Multiple  approaches  have  been  published  in  the  literature  on  2D  and  3D  wide  FOV 
ultrasound  mosaics  [63,62,64,65],  but  very  little  of  it  concerns  registering  the 
underlying  strain  held.  As  various  sources  of  decorrelation  are  usually  affecting  the 
computation  of  strain  images,  this  problem  becomes  even  more  important  when 
attempting  to  generate  a  unified,  wide  displacement  held.  Further  more,  most 
elastography  algorithms  result  in  qualitative  strain;  2  image  pairs  with  even  very 
slightly  different  degrees  of  compression  will  produce  2  strain  images  in  which 
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different  structures  could  be  visible.  Another  problem  rises  in  the  ambiguity  of  the 
interpretation  of  strain  images:  low  strain  can  be  indicative  of  high  stiffness  but  this 
interpretation  may  not  be  the  right  one  if  the  stress  field  is  not  uniform  throughout 
the  tissue  [11,38].  To  address  these  issues  and  to  improve  the  quality  of  strain 
images,  several  metrics  of  stability,  consistency /persistency  and  reliability  have  been 
developed  [30,28,66].  In  this  paper  we  propose  using  similar  techniques  to  select  a 
stable  pair  of  RF  lines  which  will  become  the  seed  for  generating  a  reliable,  wide 
FOV  displacement  field.  Displacement  on  the  seed  line  is  calculated  using  dynamic 
programming  and  later  propagated  in  both  lateral  directions  of  the  mosaic  using  an 
analytic  minimization  approach  [35].  Each  new  image  pair  adds  to  the  unified 
displacement  field.  Here  we  present  tissue  mimicking  phantom  data  for  2D 
validation.  Finally,  the  method  is  extended  to  3D  ultrasound  elastography 
mosaicing  using  multiple  3D  volume  pairs. 


4.2  Methods 

Consider  a  sequence  of  radio  frequency  (RF)  data  (Inin,  Im u)  collected  at  position 
t0,  before  and  after  the  compression  of  tissue  using  a  2D  ultrasound  transducer  (Fig. 
4.2a).  Each  sequence  is  contains  n  RF-lines  of  length  m.  A  second  sequence  ( /rn2 1 , 
Irri22)  is  collected  (Fig.  4.2b)  after  the  transducer  has  been  moved  in  the  lateral 
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direction  of  the  probe  to  position  t\,  with  the  help  of  a  moving  stage  (Fig.  4.1).  A 
vertical  stage  was  used  to  achieve  an  almost  identical  compression  rate  between  the 
two  sequences.  The  translation  between  the  2  image  sequences  is: 


1 'lines  Tmm  *  Tl/w  (4.1) 

where  w  is  the  width  of  the  ultrasound  transducer  in  millimeters  (mm),  n  is  the 
number  of  RF-lines,  Tmm  is  the  ground  truth  translation  in  mm  as  read  on  the  stage 
and  Tnnes  is  the  corresponding  translation  as  number  of  RF-lines.  The  overlap  area 
consists  of  (n  -  Tiines)  RF-lines  (Fig.  4.3). 


4.2.1  2D  Pair-wise  Mosaicing 

In  Analytic  Minimization  (AM)  elastography  [35],  2D  integer  displacements  are  first 
obtained  using  dynamic  programming  (DP)  on  a  single  pair  of  RF-lines  and  are 
later  propagated  to  produce  2D  subsample  displacements  for  the  entire  image.  We 
aim  to  find  a  robust  and  stable  seed  RF-line  in  the  overlap  area,  which  also  gives 
consistent  DP  integer  displacement  results  in  the  two  sequences.  The  displacement 
estimate  on  the  seed  fine  will  serve  as  initial  guess  in  the  AM  propagation. 
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Figure  4.1:  Experimental  setup. 


69 


CHAPTER  4.  MOSAICING 


0.5 

0 

-0.5 

-1 


1.5 

2 


2.5 


3.5 
4 

4.5 


(a)  (b) 

Figure  4.2:  Bmode  ultrasound  data  before  and  after  compression  for  (a)  position  t0, 
and  (b)  position  t\. 


Overlap  area 


Figure  4.3:  Bmode  ultrasound  data:  translation  Tnnes  and  the  length  of  the  overlap 
area. 
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4. 2. 1.1  Algorithm  Overview 

The  pairwise  strain  mosaicing  is  implemented  as  follows: 

1.  Robustness  and  stability  measures  are  used  to  identify  in  the  overlap  area  a 
suitable  seed  RF  line  for  the  mosaicing  algorithm,  similarly  to  [12].  The 
process  filters  those  candidates  which  exhibit  a  high  degree  of  decorrelation 
(Figure  4.4): 

•  k  =  5  ...,*5)  random  corresponding  positions  (RF  line  pairs)  are 

selected  from  the  overlap  area  (Fig.  4.5-1). 

•  Integer  axial  displacement  is  computed  for  each  RF  line  pair  using  the 
dynamic  programming  approach  [35]:  (a\,a\,  ...,05)  and  (a^,a|,  ...,a|) 
(Fig.  4.5-2).  Note:  in  this  step,  both  axial  and  later  integer 
displacements  are  computed,  but  from  here  on  only  axial  displacement 
values  will  be  used. 

•  In  a  continuous  piece  of  tissue,  the  axial  displacement  profile  resulting 
from  a  stress  field  induced  by  applied  compression  has  a  monotonous 
ramp  [59] .  A  pair  of  decorrelated  RF  lines  would  affect  the  DP 
displacement  estimation  algorithm.  Instead  of  a  globally  optimal 
solution,  the  estimate  could  exhibit  regions  of  locally  optimal  solutions. 
For  small  deformations,  these  locally  optimal  solutions  would  correspond 
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to  perturbations  in  the  monotonously  decreasing  slope  of  their 
displacement  profile,  which  in  turn  could  result  in  artifacts  in  the  final 
strain  image.  If  three  or  more  positions  along  the  displacement  profile 
would  exhibit  a  continuous  change  in  the  slope  of  displacement,  this 
could  point  to  a  region  of  poorly  correlated  data. 

•  For  the  remaining  positions  which  satisfy  the  monotonous  slope  criterion, 

dk  =  abs(a\  —  avgk  =  mean(dk);  stdk  =  stdev(dk )  (4.2) 

We  select  the  seed  RF  line  s  as  the  position  with  the  minimum  stdk  and 
avgk  <  1. 

•  At  the  end  of  the  subinteger  displacement  estimation  process  on  the  a 
pair  of  RF  lines,  (A,  =  a,  +  A  af),  (Lt  —  lt  +  A  If)  are  the  axial  and  lateral 
displacements  at  each  sample  i.  The  magnitude  of  the  normalized 
cross-correlation  could  be  used  to  assess  the  degree  of  matching  between 
the  pre-  and  post-compression  data. 

1(-^l(b  8)  ~  if  +  Aj,  S  +  Lf)  —  I 2) 

-  hyffffiMi  +  A,,*  +  li)  -  /2p 

(4.3) 

where  I\  and  /2  are  the  means  of  RF  values  along  the  seed  line  s.  The 
value  of  C  could  be  used  as  an  additional  check  to  verify  if  the 
displacement  estimate  for  the  selected  seed  line  is  a  good  fit.  If  C  value 
would  fall  below  a  certain  threshold  (to  be  established  by  the  user),  the 
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seed  line  would  be  deemed  not  suitable  and  the  next  stable  seed  line 
would  be  selected  (Fig.  4.5-3). 

2.  Subinteger  displacements  values  are  calculated  for  seed  line  s  (Fig.  4.5-4). 

3.  The  subinteger  displacement  values  are  propagated  using  AM: 

•  in  one  sequence  towards  the  first  RF  line,  and] 

•  in  the  second  sequence  towards  the  last  RF  line.] 

The  unified  displacement  field  will  have  n  +  Tunes  RF  lines  and  the  stitch  will 
be  around  RF  line  number  s  +  Tunes  (Fig.  4.5-5). 

4.  Generate  the  ultrasound  elastography  mosaic  from  the  unified  displacement 
field  (Fig.  4.5-6). 


4.2.2  2D  Multi-image  Mosaicing 


Given  an  existing  nt  wide  displacement  map,  computed  for  image  pairs  1  through  t, 
we  expand  our  displacement  map  to  include  information  from  image  pair  t  +  1  as 
follows: 
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Figure  4.4:  Robust  seed  selection  method. 
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Figure  4.5:  Pairwise  Mosaicing  workflow:  k  =  5  random  RF  positions  are  selected 
(1),  integer  displacement  is  calculated  for  each  position  (2),  the  seed  RF  line  s  is 
selected  (3),  subinteger  displacement  values  are  calculated  for  s  (4),  propagated  in 
the  2  sequences  and  stitched  (5).  The  ultrasound  elastography  mosaic  is  generated 
from  the  unified  displacement  field  (6). 
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4. 2. 2.1  Algorithm  Overview 

1.  Compute  the  translation  7f^a  between  position  t  and  position  t  +  1  and  the 
overlap  area. 

2.  Select  the  seed  RF-line  st+i  in  the  new  overlap  area. 

3.  Use  the  subinteger  values  form  the  wide  FOV  displacement  map  at  RF-line 
number  st+i  +  Tunes  as  seed  values  to  propagate  using  AM: 

•  in  the  new  sequence  towards  the  first  RF-line. 

Stitch  the  newly  computed  (st+i  +  Tunes )  lines  at  the  beginning  of  the  previous 
displacement  map.  The  new  mosaiced  displacement  field  will  have  nt  +  7f^s 
RF  lines. 

4.  Generate  the  new  ultrasound  elastography  mosaic  from  the  updated 
displacement  field. 


4.2.3  3D  Elastography  Mosaicing 

In  the  three-dimenssional  case,  consider  a  pair  of  RF  volumes  (Un,  Vi. 2)  collected  at 
position  before  and  after  the  compression  of  tissue  using  a  3D  ultrasound 
wobbler  transducer  (NEED  FIGURE  OF  WOBBLER  HERE).  Each  volume 
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contains  k  frames  and  each  frame  has  n  RF  lines  of  length  m.  One  or  more 
subsequent  pairs  of  volumes  are  collected  after  the  wobbler  transducer  is  moved  in 
the  lateral  direction  of  the  probe  with  the  help  of  a  moving  stage.  Once  again,  a 
vertical  stage  was  used  to  achieve  an  almost  identical  compression  rate  between  the 
two  sequences  (Figure  4.1).  Using  the  lateral  and  axial  stages  ensures  the  first  frame 
of  the  first  volume  pair  is  alligned  with  the  first  frame  of  each  subsequent  pair 
volume,  and  all  the  other  frames  are  simillarly  aligned,  with  no  out-of-plane  motion 
between  each  position.  The  same  steps  of  multi-image  mosaicing  can  be  applied  on 
a  frame  by  frame  basis,  resulting  in  an  unified  displacement  field  for  each  elevational 
position  of  the  wobbler  motor. 

Note:  here  we  employed  a  fixed  rigid  setup  to  controll  the  acquizition  of  3D 
ultrasound  data  for  the  purpose  of  building  3D  strain  mosaics.  In  a  routine  3D 
ultrasound  exam,  using  stages  to  control  the  imaging  of  a  certain  human  tissue  is 
absolutely  not  feasibile.  This  does  not,  however  invalidate  our  framework.  As 
electromagnetic  trackers  become  integrated  into  ultrasound  transducers,  it  is 
feasibile  to  select  ultrasound  volumes  consisting  only  of  aligned  frames  with  no 
out-of-plane  rotation.  The  work  presented  here  makes  a  case  for  in-plane  mosaicing 
where  translation  is  the  only  component  of  the  transformation.  One  can  easily 
envision  an  ultrasound  system  equipped  with  a  tracked  transducer  like  in  the 
Foroughi  set-up  [?]. 
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4. 2. 3.1  Algorithm  Overview 

Here  we  used  a  controlled  setup  for  proof  of  concept,  but  using  the  Foroughi 
method  [?]  one  could  achieve  a  3D  elastography  mosaic  as  follows: 

1.  Perform  freehand  palpation  with  tracked  3D  transducer, 

2.  Use  the  tracking  information  [?]  to  select  suitable  aligned  frames  separated 
only  by  in-plane  translation, 

3.  Apply  2D  multi- image  mosaicing  method  on  a  frame-by-frame  basis  to 
estimate  an  unified  displacement  field,  and 

4.  Generate  the  3D  elastography  mosaic  from  the  resulting  displacement  field. 

4.3  2D  Mosaicing  Experimental  Setup 

For  experimental  validation  we  palpated  a  CIRS  (Norfolk,  VA)  elastography 
phantom  049a  (Figure  4.1)  using  a  high-frequency  ultrasound  transducer 
(L14-5W/60)  at  center  frequency  of  10  MHz.  Ultrasound  RF  data  was  acquired 
from  an  Ultrasonix  system  (Vancouver,  BC)  at  40MHz  sampling  rate.  The  049a 
CIRS  phantom  consists  of  a  series  of  stepped  cylinders  of  varying  diameters  (NEED 
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(e)  (f)  (g) 

Figure  4.6:  Ultrasound  elastography  at  position  t0  (a),  U  (b),  t2  (c),  t3  (d)  and 
elastography  mosaic  of  positions  t0  and  U  (e),  t0,  t\  and  t2  (f),  and  t0,  U,  t2  and  t3 

(§)• 

PICTURES  FROM  THE  CIRS  BROCHURE).  The  transducer  was  placed  on  top  of 
one  of  the  cylinders,  parallel  with  its  direction.  The  phantom  was  placed  on  a  stage 
which  controlled  the  compression  in  the  axial  direction  and  the  translation  in  the 
lateral  direction  (Figure  4.1).  RF  data  sequences  were  acquired  from  four  axial 
compression  levels  (c0  =  0,  C\  =  2.54mm,  c2  =  5.08mm,  c3  =  back  to  0),  for  each  of 
four  lateral  translation  positions  of  the  transducer  (t0  =  0,  U  =  5.08mm,  t2  = 
10.16mm,  t3  =  20.32mm),  for  a  total  of  sixteen  data  sets.  The  width  of  the 
ultrasound  transducer  L14-5W  is  59mm,  and  as  a  consequence  any  2  sets  separated 
by  a  lateral  translation  had  some  amount  of  overlap. 

Ultrasound  elastography  images  were  obtained  for  each  translation  position  t0  -  h 
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width  (mm)  width  (mm) 


(d)  (e)  (f) 

Figure  4.7:  Panoramic  displacement  map  of  positions  to  and  to  (a),  and  of  positions 
to,  G,  t2  and  to  (b),  the  absolute  difference  between  them  (c),  and  the  mean  and 
standard  deviation  of  this  difference,  per  line  (f).  The  unit  of  displacement  is  pixels. 
Corresponding  ultrasound  elastography  mosaics  of  positions  t0  and  t3  (d),  and  t0,  G, 
t2  and  t:i  (e). 


((Fig.  4.6a-d).  A  pairwise  elastography  mosaic  was  produced  from  positions  t0  and 
G  (Fig.  4.6e),  and  then  multi-image  elastography  mosaics  from  positions  to,  G  and 
t2  (Fig.  4.6f),  and  t0,  G,  t2  and  t3  (Fig.  4.6g). 
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4.3.1  Phantom  Results 

Figure  4.7  shows  the  results  of  the  validation  experiment.  The  two  panoramic 
displacement  maps  agree  almost  exactly;  the  absolute  difference  for  the  entire  image 
had  a  mean  of  0.0755  and  a  standard  deviation  of  0.2386  pixels.  The  maximum 
difference  was  2.0168  pixels  and  it  did  not  exceed  1  pixel  on  a  mean,  per-line  basis. 
For  a  metric  correspondent,  2.0168  pixels,  is  the  equivalent  of  0.0406  mm  in  the  axial 
direction.  Once  the  displacement  maps  are  converted  in  elastogram  mosaics  (Fig. 

4.7  e,f),  the  difference  becomes  indistinguishable  even  on  close  inspection.  With  the 
clinical  application  in  mind,  we  conclude  that  these  small  differences  are  acceptable. 


4.4  3D  Mosaicing  Experimental  Setup 

For  3D  validation,  RF  data  was  acquired  using  an  Ultrasonix  system  (Vancouver, 
BC)  at  20MHz  sampling  rate.  The  same  CIRS  (Norfolk,  VA)  phantom  was  palpated 
with  a  38mm  width  linear  4D  volumetric  transducer  (4DL14-5/38)  (Figure  FIG  OF 
WOBBLER).  As  with  the  2D  experiment,  the  phantom  was  placed  on  a  stage  to 
controll  the  compression  in  the  axial  direction  and  the  translation  in  the  lateral 
direction  (Fig.  4.1).  RF  data  sequences  were  acquired  from  four  axial  compression 
levels  (c0  =  0,  Ci  =  2.54mm,  c2  =  5.08mm,  c3  =  back  to  0),  for  each  of  five  lateral 
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Figure  4.8:  3D  ultrasound  elastography  mosaic:  axial-lateral  plane  (a),  lateral- 
elevational  plane  (b),  and  3  axial-elevational  planes  through  3  different  diameters 
of  the  stepped  cylinder  (c,  d,  e) 


translation  positions  of  the  transducer  (t0  =  0,  t\  =  5.08mm,  t2  =  10.  f 6mm,  f3  = 
20.32mm,  f4  =  22.86mm),  for  a  total  of  twenty  data  sets. 
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4.4.1  Phantom  Results 

Figure  4.8  shows  the  results  of  the  3D  ultrasound  elastography  mosaic.  Strain  in  the 
middle  frames  of  the  axial-lateral  plane  (Figure  4.8a),  lateral-elevational  plane 
(Figure  4.8b),  and  3  axial-elevational  planes  (Figure  4.8c-e)  is  shown.  The  diameters 
of  the  stiff  cyclinder  along  the  mosaicing  direction  were:  4:05  mm  (Figure  4.8c),  2:53 
mm  (Figure  4.8d)  and  1:58  mm  (Figure  4.8e). 


4.5  Conclusion  and  Future  Work 

We  have  presented  an  algorithm  for  generating  reliable  multi-image  ultrasound 
elastography  mosaics,  robust  to  regions  of  decorrelation.  Panoramic  B-mode 
ultrasound  has  been  reported  with  success  in  the  literature  and,  with  the  addition  of 
corresponding  elastograms,  an  ultrasound  system  with  these  capabilities  has  great 
potential  in  clinical  diagnostic  and  monitoring.  Elastography  has  the  advantage 
that  it  requires  no  additional  hardware  to  be  implemented,  and  a  majority  of 
commercial  ultrasound  systems  now  present  an  elastography  interface.  Furthermore, 
the  challenge  of  mosaicing  while  freehand  scanning  can  be  met  as  efforts  are  under 
way  to  incorporate  electromagnetical  (EM)  tracking  in  ultrasound  transducers. 
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Ultrasound  Elastography  as  a  Tool 
for  Imaging  Guidance  During 
Prostatectomy:  Initial  Experience 


A.l  Clinical  Significance 


Prostate  cancer  is  the  second  leading  cause  of  cancer  death  and  the  most  common  can¬ 
cer  detected  in  men  in  the  United  States.  An  estimated  217,730  new  cases  of  prostate 
cancer  were  diagnosed  in  the  United  States,  and  approximately  32,050  men  died  of 
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prostate  cancer  during  2010  [67].  Radical  Prostatectomy  (RP)  aims  for  complete  can¬ 
cer  resection  and  has  been  shown  to  improve  cancer  survival  [68].  Robotic- assisted 
laparoscopic  prostatectomy  (RALP)  has  recently  emerged  as  an  alternative  to  open 
and  laparoscopic  procedures.  The  daVinci  Surgical  System  (Intuitive  Surgical,  Sunny¬ 
vale,  CA)  provides  3-D  visualization,  higher  magnification,  hand  tremor  elimination 
and  refined  dexterity  by  incorporating  wristed  instrumentation.  From  250  robotic 
cases  in  the  beginning  (2001),  the  number  has  reached  73,000  in  2009  (86  %  of  the 
85,000  American  men  who  had  prostate  cancer  surgery)  [69,70]. 

Initial  experiences  with  the  daVinci  surgical  system  have  been  positive:  short  learn¬ 
ing  curve,  limited  blood  loss,  less  post-operative  pain,  favorable  complication  rates, 
and  short  hospital  stay  [69,70,71,72,73,74,75,76].  Despite  fewer  perioperative  com¬ 
plications  and  shorter  hospital  stay,  a  recent  paper  found  patients  were  three  times 
more  likely  to  require  salvage  therapy  [77] .  One  theoretical  disadvantage  with  regards 
to  robotic  surgery  is  the  lack  of  tactile  feedback.  In  open  RPs,  the  surgeon  uses  his 
fingers  to  feel  the  periphery  of  the  prostate  gland  [78].  Without  tactile  feedback,  a 
robotic  surgeon  is  unable  to  appreciate  differences  in  tissue  texture  or  firmness  and 
therefore  may  not  be  able  to  tailor  precisely  the  extent  of  tissue  excision  around 
the  prostate  gland  in  efforts  to  eradicate  all  cancerous  tissue.  Inadvertently  leaving 
residual  cancer  cells  behind,  called  a  positive  surgical  margin  (PSM),  is  highly  asso¬ 
ciated  with  cancer  recurrence.  PSM  rates  were  initially  higher  in  RALP  than  in  the 
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open  procedure,  but  they  have  been  shown  to  decrease  with  surgeon’s  experience  and 
improved  technique  [75,77]. 

As  manual  palpation  helps  guide  the  surgeon  in  the  open  procedure,  an  equivalent 
real-time  guiding  tool  is  needed  for  robotic  prostatectomy.  Imaging  modalities  like 
MRI  or  CT  are  not  feasible  intraoperatively,  nor  do  they  possess  the  sensitivity  or 
specificity  for  accurate  detection  and  localization  of  prostate  cancer.  Transrectal  ul¬ 
trasound  (TRUS)  is  routinely  used  in  diagnosis,  in  conjunction  with  digital  rectal 
examination  (DRE)  and  biopsies  [79].  One  center  used  TRUS  for  real-time  moni¬ 
toring  and  guidance  during  Laparoscopic  RP  and  reported  technical  feasibility  and 
enhanced  precision  by  decreased  PSM  rates  [80,81].  TRUS  was  capable  of  imaging 
a  substantial  percent  of  nonpalpable  prostate  cancers.  The  authors  recognized  how¬ 
ever,  the  limitations  of  TRUS  guidance;  it  requires  considerable  prior  expertise  and 
tends  to  identify  primarily  hypoechoic  lesions,  which  were  just  47%  of  the  cancer 
nodules  studied  [80] .  Today’s  prostate  cancer  patients  are  more  likely  to  present  with 
echogenic  or  isoechoic  lesions  because  aggressive  screening  techniques  lead  to  a  shift 
toward  smaller,  early-stage  cancers  [82,  83]  ;  classic  B-mode  gray- scale  ultrasound 
alone  cannot  identify  these  lesions. 

Ultrasound  (US)  Elastography  (USE)  is  emerging  as  a  valuable  tool  in  the  field  of 
imaging.  Elastography  is  a  qualitative  technique  based  on  the  principle  that  tissue 
compression  produces  strain  (displacement)  within  that  tissue;  strain  is  smaller  in 
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harder,  stiffer  tissue  than  in  softer,  more  compliant  tissue  11.  Analyzing  the  ultra¬ 
sound  raw  radio  frequency  signal  results  in  a  strain  map,  commonly  called  elastogram, 
where  harder  tissue  is  darker  than  surrounding  soft  tissue.  Cancers  tend  to  present 
as  hard  lesions  due  to  increased  cellularity  [11].  Echogenicity  and  stiffness  of  tissue 
are  generally  uncorrelated;  USE  can  identify  hypoechoic  lesions,  but  also  echogenic 
or  isoechoic  cancers  that  classic  gray-scale  ultrasonography  cannot.  Elastography 
through  the  transrectal  approach  has  already  been  proven  feasible  in  guiding  biopsies 
of  the  prostate  [84,85,86,87].  Integrating  USE  technology  with  a  laparoscopic  ultra¬ 
sound  probe  will  give  robotic  and  laparoscopic  surgeons  an  important  image-guidance 
tool,  which  until  this  point  does  not  exist  [88,89,90]. 

Here  we  describe  the  methodology  and  results  of  a  pilot  study  evaluating  the  diag¬ 
nostic  accuracy  and  efficacy  of  using  USE  to  identify  the  cancerous  nodules  in  the 
prostate  gland.  In  the  study,  human  ex-vivo  prostatectomy  specimens  where  used 
to  assess  the  accuracy  of  USE  in  the  identification  and  characterization  of  hard  can¬ 
cerous  nodules.  We  compared  the  elastogram  results  with  histopathology  maps  (the 
gold  standard)  and  also  to  pre-  and  post-operative  MR  scans  of  the  prostate  gland  in 
order  to  assess  and  co-localize  anatomically  USE  with  the  reference  histopathology 
and  MR  scans. 
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A. 2  Materials  and  Methods 


Right  Urethra 


Left 


Figure  A.l:  Ultrasound  elastography  data  collection  process  using  the  sextant  ap¬ 
proach;  RF  data  was  acquired  in  axial  planes  (1-6)  from  the  gland’s  base  towards 
the  apex.  For  illustration  purposes,  a  lesion  is  outlined  in  the  left  mid  section,  pe¬ 
ripheral  zone  of  the  specimen,  similar  with  the  case  of  specimen  #3 


Prostate  cancer  patients,  candidates  for  prostatectomy,  were  prospectively  enrolled  in 
the  study,  following  an  informed  consent  approved  by  the  Institutional  Review  Board. 
The  objective  of  the  study  was  to  evaluate  the  efficacy  of  using  elastography  to  identify 
and  precisely  localize  hard  nodules  such  as  seen  with  prostate  cancer  just  beneath  the 
surface  of  the  prostate  gland  in  the  peripheral  zone.  In  this  area,  cancerous  lesions  are 
at  most  risk  of  invasion  beyond  the  confines  of  the  prostate  gland  and  also  more  likely 


to  be  cut  across  by  a  well  meaning  surgeon.  We  recruited  patients  who  underwent 
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both  open  and  assisted  prostatectomies  given  that  the  process  of  removing  the  gland 
was  not  a  focus  of  our  study.  Patients  underwent  multiple  radiological  procedures. 
1)  Pre-operative  3  Tesla  MRI  of  the  pelvis  was  performed  right  before  the  surgery 
procedure.  2)  Post-operative  ultra  high-resolution  MRI  at  9.4  Tesla  was  performed 
on  the  excised  prostate  specimen  to  correlate  the  results  to  in-vivo  pre-operative 
imaging.  3)  USE  was  then  performed  on  the  prostate  specimen  by  an  experienced 
radiologist  blinded  to  the  surgeon’s  findings  and  to  the  pre-operative  pathology  report. 
The  collected  radio- frequency  (RF)  data  was  used  offline  to  recreate  classic  B-mode 
grey-scale  images,  and  also  to  compute  elastograms  showing  the  stiffness  of  the  tissue 
scanned  [34,35]. 

Pre-operative  and  post-operative  MR  scans  were  used  for  anatomical  correlation  with 
the  computed  elastograms.  For  USE,  the  prostate  specimens  were  placed  in  prone 
position  on  a  surgical  table.  USE  scans  were  performed  in  a  systematic  sextant 
approach,  similar  to  that  used  for  image  guided  biopsies.  RF  data  was  acquired  in 
axial  planes  (from  gland’s  base,  through  mid  gland,  to  apex)  on  the  left  and  right 
side  of  the  gland  (Figure  A.l).  The  sextant  approach  was  necessary  to  ensure  that 
the  scans  were  in  the  same  plane  with  the  histopathology  diagrams  (axial)  which 
constituted  the  gold  standard  for  comparison.  USE  coronal  scans  from  the  left  to  the 
right  of  the  gland  were  also  performed;  these  scans  were  in  alignment  with  the  MR 
coronal  scans. 
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Hardware  and  Software  Specification 

USE  acquisition  was  conducted  using  a  Siemens  Antares  US  scanner  (Siemens  Med¬ 
ical  Solutions  USA,  Inc.  Ultrasound  Division,  Issaquah,  WA)  with  an  ultrasound 
research  interface  to  access  raw  RF  data.  Data  was  acquired  by  manual  handling 
using  a  Siemens  VF  10-5  linear  array  for  prostate  specimens.  After  RF  data  collec¬ 
tion,  elastograms  were  obtained  using  the  dynamic  programming  (DP)  elastography 
algorithm  developed  by  Rivaz  et  al  [34, 35] . 

Each  prostate  specimen  underwent  routine  pathologic  processing  and  analysis.  Due  to 
the  high  volume  of  prostatectomies  performed  at  our  institution,  the  routine  patholog¬ 
ical  process  does  not  result  in  a  whole  mount  mapping.  Instead,  for  histopathological 
evaluation  the  prostatectomy  specimens  were  initially  sliced  at  every  3-4  mm  from 
apex  to  base,  according  to  the  Stanford  protocol.  Each  slice  (6  to  10  master  slices) 
were  then  incorporated  in  a  paraffin  block  and  sliced  at  5  n  meter  thickness.  The 
slices  were  stained  with  hematoxylin-eosin  and  were  then  analyzed  under  a  micro¬ 
scope  by  a  pathologist  blinded  to  the  surgeon’s  findings  and  also  to  the  elastography 
results.  The  localization  and  size  of  each  tumor  focus  were  documented  for  all  step 
master  slices  on  axial  diagrams,  with  Gleason  score.  Large  macro  photographs  were 
reconstructed  in  several  specimens  (Figure  4.6(c)).  All  data  collected  were  stored  in 
the  database. 

N  =  10  target  areas  were  analyzed  from  N  =  6  patients  enrolled  so  far  into  the  elas- 
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tography  analysis.  Histological  findings  served  as  the  gold  standard  in  determining 
the  presence,  location  and  size  of  any  prostatic  nodules,  malignant  and  benign.  The 
objective  of  our  study  was  then  to  compare  axial  elastograms  findings  with  the  his¬ 
tological  findings  recorded  by  the  pathologist  (mapping  diagrams,  measurements  and 
nodule  characteristics  such  as  malignant  vs.  benign).  Since  histopathology  diagrams 
often  specified  just  the  maximum  diameter  of  a  lesion,  coronal  elastograms  were  used 
to  better  establish  the  location  and  extent  of  the  identified  lesions.  MRI  images  (both 
axial  and  coronal  planes)  were  aligned  to  the  elastograms  and  provided  help  with  their 
anatomical  co-registration  using  anatomical  details  such  as  urethra  or  boundaries  of 
peripheral  zone  vs.  central  gland. 


A. 3  Results 

Elastography  identified  N=I0  lesions,  8  hard  nodules  in  the  peripheral  zone,  1  hard 
and  1  soft  nodule  in  the  central  gland  (Table  1).  Pathology  reports  showed  8  of  these 
lesions  as  malignant  and  2  as  benign.  Diameter  measurements  correlation  proved 
difficult  because  of  the  inability  to  perfectly  register  the  three  investigative  modalities. 
USE  and  MRI  measurements  were  within  on  average  2.05  mm  vs.  2.25  mm  of  the 
diameters  measured  by  pathology  (standard  deviation  of  1.9  mm  for  USE  and  2.9 
mm  for  MRI).  Size  measurements  and  Gleason  score  are  reported  in  Table  1. 
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Table  A.l:  Prostate  specimen  data:  A  total  of  10  (ten)  elastography  lesions  were 
identified  in  6  (six)  patients  specimens  (8  malignant  and  2  benign).  PZ  =  peripheral 
zone,  CG  =  central  gland 


# 

Location 

Gleason 

Size  (cm) 

Score 

Elastography 

Pathology 

MRI 

1.1 

PZ  base 

3+5 

1.4  x  0.8 

1.3  x  0.8 

1.3  x  1.1 

1.2 

CG  base 

N/A-Solid 

0.7  x  1.1 

1.0  x  1.0 

1.0  x  1.1 

1.3 

CG  base 

N/A-Soft 

1.1  x  0.8 

1.0  x  1.0 

1.0  x  0.9 

2.1 

PZ  base 

5+3 

3.0  x  1.3 

2.4  x  1.0 

2.0  x  1.5 

3.1 

PZ  mid 

4+5 

2.4  x  0.8 

1.9  x  1.0 

1.5  x  1.2 

4.1 

PZ  mid 

3+3 

1.0  x  0.5 

0.5  x  0.4 

0.6  x  0.7 

4.2 

PZ  mid 

3+4 

1.5  x  0.9 

1.1  x  0.5 

1.1  x  0.8 

5.1 

PZ  apex 

3+3 

0.5  x  0.6 

0.5  x  0.5 

0.6  x  0.6 

5.2 

PZ  apex 

4+3 

0.6  x  1.0 

0.8  x  0.9 

0.9  x  0.9 

6.1 

PZ  base 

3+3 

0.7  x  1.2 

0.7  x  1.8 

0.7  x  0.7 
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Specimen  #1  presented  multiple  hard  and  soft  lesions,  located  in  the  central  gland 
of  the  prostate  (Figure  A. 2).  The  ex-vivo  T2- weighted  coronal  image  from  specimen 
MRI  obtained  at  9.4  Tesla  (Figure  A. 2  (C)  -  here  in  counter  clock  wise  orientation  for 
better  visualization  of  the  correlation  between  USE  and  MRI  of  the  specimen)  shows 
detailed  anatomy  of  the  heterogeneous  central  gland  with  a  solid  benign  prostatic 
hypertrophy  nodule  (BPH)  confirmed  by  pathology.  Elastography  was  able  to  detect 
this  solid  nodule  despite  the  heterogeneity  of  the  prostate  (Figure  A. 2  (B))  -  solid 
arrow,  whereas  the  lesion  was  not  clearly  identified  by  gray  scale  ultrasound.  Ex-vivo 
T2-weighted  9.4  Tesla  coronal  image  from  specimen  MRI  also  shows  an  additional 
soft  cystic  BPH  nodule  (dashed  arrow).  Urethra  is  also  visible  on  the  elastogram  ,  as 
well  as  MRI  exam  (labeled  urethra ). 

Axial  scans  of  the  same  specimens  were  compared  with  histopathology  axial  cross- 
sections.  The  prostate,  submitted  for  histological  processing  in  four  quadrant  sections 
per  slice,  was  digitally  realigned  to  reconstruct  a  full  histological  cross-section  (Figure 
A. 3(c)).  Specimen  #1  was  found  with  a  tumor  with  Gleason  score  of  8  at  the  prostate 
base,  left  side  (outlined).  Figure  A. 3(a)  shows  an  ultrasound  B-mode  image  and  an 
elastogram  obtained  through  an  axial  plane  at  the  prostate’s  base  on  the  left  side. 
The  same  tumor  was  identified  by  elastography  (Figure  A. 3(b)-  dashed  contour)  but 
is  not  visible  on  grey-scale  ultrasound.  For  an  anatomical  correlation,  a  soft  -  cystic 
nodule  anterior  to  cancer  can  be  seen  on  B-mode  image  and  USE  (Figure  A. 3(a),  (b) 
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Figure  A. 2:  Coronal  section  of  prostate  specimen  #1  at  the  level  of  the  central  gland. 
Classic  ultrasound  B-mode  (A)  and  elastogram  (B).  9.4  Tesla  ex-vivo  (C)  and  3  Tesla 
in-vivo  (D)  MRI  images  are  presented  in  coronal  planes,  in  CCW  (counter  clock  wise) 
orientation  for  better  visualization  of  the  correlation  between  USE  and  MRI  of  the 
specimen.  Benign  solid  (arrow)  and  soft  (dashed  arrow)  nodules  and  urethra  are 
visible. 
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-  arrows)  and  on  histopathology  (Figure  A. 3(c)  -  arrows). 


The  remaining  5  (five)  prostate  specimens  presented  with  superficial  lesions  in  the 
peripheral  zone  of  the  gland.  USE  identified  multiple  hard  malignant  lesions  in  various 
locations,  from  the  base  to  the  apex  of  the  prostate  gland.  One  can  notice  the  clear 
delimitations  of  these  lesions  on  USE  (Figure  A. 4)  as  well  as  the  close  estimations 
of  size  versus  pathology  and  MRI. 


Figure  A. 3:  Axial  section  of  prostate  specimen  #1  peripheral  zone.  Left  lateral 
section  of  the  prostate’s  base;  classic  ultrasound  B-mode  (a)  and  elastogram  (b). 
Hard  lesion  is  outlined,  arrows  point  to  adjacent  nodule,  (c)  Hematoxylin  &  eosin 
stained  histological  section  of  prostate  base.  The  tumor  (Gleason  score  3+5  =  8, 
outlined  in  black)  extended  beyond  the  prostatic  capsule  in  this  section  and  invaded 
the  left  seminal  vesicle  (arrow). 
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Figure  A. 4:  B-mode  image  (left)  and  elastogram  (right)  from  specimens  #  2-6.  Dark 
regions  at  the  very  bottom  of  elastograms  represent  structures  outside  the  prostate 
tissue  (e.g.  operating  table).  The  border  of  the  prostatic  tissue  can  be  easily  noticed 
as  a  highly  reflective  band  at  the  bottom  of  B-mode  images. 
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A. 4  Discussion 

In  surgical  procedures  where  manual  palpation  would  be  helpful  but  not  possible  to 
perform,  e.  g.  laparoscopic  robotic  surgery,  USE  can  offer  added  value  if  proven  to 
be  accurate  in  detecting  pathologic  lesions.  Our  feasibility  study  showed  that  USE 
was  able  to  identify  both  hard  and  soft  lesions  in  the  ex-vivo  prostate  specimens,  lo¬ 
cated  in  the  deep  prostatic  central  gland  and  in  the  peripheral  zone.  Histopathologic 
findings  validated  USE,  and  results  compared  favorably  with  the  in-vivo  pre-surgical 
and  ex-vivo  post-surgical  MRI  scans.  In  the  central  gland  of  the  prostate,  elastogra- 
phy  showed  excellent  detection  of  hard  and  soft  areas,  despite  the  complexity  of  the 
central  gland.  Elastography  was  able  to  identify  both  hard  and  soft  BPH  nodules 
and  anatomical  landmarks  like  the  urethra  (note  excellent  anatomical  correlation  to 
MR  scan  findings).  In  the  peripheral  zone  USE  identified  multiple  hard  malignant 
lesions,  from  the  base  to  the  apex  of  the  prostate  gland.  These  preliminary  results 
demonstrate  the  ability  of  USE  to  detect  hard  nodules  in  the  prostate  and  are  en¬ 
couraging  in  the  pursuit  of  this  technology  as  a  palpation  equivalent  imaging  tool  for 
prostatectomy. 

USE  maps  tissue  elasticity  which  makes  it  an  ideal  imaging  modality  to  serve  as  a 
surrogate  and  possible  equivalence  to  manual  palpation  in  identifying  hard  cancer¬ 
ous  tissue  in  the  prostate  gland,  especially  in  the  peripheral  zone  but  also  in  the 
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central  gland.  Real-time  intra-operative  imaging  guidance  is  needed  for  identifying 
the  presence  of  cancer  within  the  prostate,  especially  near  the  capsule  where  tu¬ 
mor  can  invade  and  spread  outside  of  the  gland,  and  also  for  studying  surrounding 
structures.  If  diseased  hard  lymph  nodes  could  be  detected,  then  lymphadenectomy 
may  provide  a  more  accurate  cancer  staging,  help  tailor  future  therapy,  and  poten¬ 
tially  prevent  recurrence.  A  better  delineation  of  the  bladder  neck  and  apex  during 
dissection,  especially  when  prostate  cancer  is  located  at  the  apex  could  perhaps  im¬ 
prove  patients’  outcome.  If  deemed  possible,  imaging  cavernous  nerves  (CNs)  located 
along  the  immediate  surface  of  the  prostate  gland  may  lead  to  their  preservation,  and 
thus  improved  preservation  of  potency  and  urinary  continence  [91].  Further  more, 
the  development  of  the  elastography  technology  as  an  imaging  guiding  tool  during 
prostatectomy  could  potentially  be  useful  in  the  open  procedures  as  well,  where  the 
manual  palpation  would  not  be  enough  to  identify  deeper  lesions.  It  has  been  docu¬ 
mented  in  the  literature  that  prostate  carcinoma  originates  in  the  central  gland  and 
transitional  zone  in  up  to  30%  of  cases  [92,93]. 


A. 5  Conclusions 

Our  initial  experience  showed  USE  was  able  to  reliably  identify  hard  nodules  in  the 
peripheral  zone  of  the  prostate  that  were  prostate  cancers.  Additionally,  USE  showed 
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its  ability  to  define  tissue  hardness  of  BPH  nodules  despite  the  underlying  tissue 
complexity  in  the  central  gland.  Our  initial  experience  with  USE  encourages  us  to 
pursue  further  the  evaluation  of  this  technique.  Further  testing  of  a  laparoscopic 
ulrasound  probe  is  needed  in  a  real  laparoscopic  environment.  We  can  conclude  that 
there  is  promise  is  integrating  laparoscopic  ultrasound  elastography  as  a  real-time, 
in-vivo  imaging  tool  to  guide  surgeons  during  robotic-assisted  prostatectomies. 
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Background: 

Summary 

During  laparoscopic  or  robotic  assisted  laparoscopic  prostatectomy,  the  surgeon  lacks  tactile  feed¬ 
back  which  can  help  him  tailor  the  size  of  the  excision.  Ultrasound  elastography  (USE)  is  an 
emerging  imaging  technology  which  maps  the  stiffness  of  tissue.  In  the  paper  we  are  evaluating 
USE  as  a  palpation  equivalent  tool  for  intraoperative  image  guided  robotic  assisted  laparoscopic 
prostatectomy. 

Material/Methods: 

Two  studies  were  performed:  1)  A  laparoscopic  ultrasound  probe  was  used  in  a  comparative  study 
of  manual  palpation  versus  USE  in  detecting  tumor  surrogates  in  synthetic  and  ex-vivo  tissue  phan¬ 
toms;  N=25  participants  (students)  were  asked  to  provide  the  presence,  size  and  depth  of  these 
simulated  lesions,  and  2)  A  standard  ultrasound  probe  was  used  for  the  evaluation  of  USE  on  ex- 
vivo  human  prostate  specimens  (N=10  lesions  in  N=6  specimens)  to  differentiate  hard  versus  soft 
lesions  with  pathology  correlation.  Results  were  validated  by  pathology  findings,  and  also  by  in-vivo 
and  ex-vivo  MR  imaging  correlation. 

Results: 

In  the  comparative  study,  USE  displayed  higher  accuracy  and  specificity  in  tumor  detection  (sen- 
sitivity=84%,  specificity=74%).  Tumor  diameters  and  depths  were  better  estimated  using  USE  ver¬ 
sus  with  manual  palpation.  USE  also  proved  consistent  in  identification  of  lesions  in  ex-vivo  pros¬ 
tate  specimens;  hard  and  soft,  malignant  and  benign,  central  and  peripheral. 

Conclusions: 

USE  is  a  strong  candidate  for  assisting  surgeons  by  providing  palpation  equivalent  evaluation  of 
the  tumor  location,  boundaries  and  extra-capsular  extension.  The  results  encourage  us  to  pursue 
further  testing  in  the  robotic  laparoscopic  environment. 
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Background 


Prostate  cancer  is  the  second  leading  cause  of  cancer  death 
and  the  most  common  cancer  detected  in  men  in  the 
United  States.  An  estimated  217,730  new  cases  of  prostate 
cancer  were  diagnosed  in  the  United  States,  and  approx¬ 
imately  32,050  men  died  of  prostate  cancer  during  2010 

[1] .  Radical  Prostatectomy  (RP)  aims  for  complete  cancer 
resection  and  has  been  shown  to  improve  cancer  survival 

[2] .  Robotic-assisted  laparoscopic  prostatectomy  (RALP) 
has  recently  emerged  as  an  alternative  to  open  and  laparo¬ 
scopic  procedures.  The  daVinci  Surgical  System  (Intuitive 
Surgical,  Sunnyvale,  CA)  provides  3-D  visualization,  high¬ 
er  magnification,  hand  tremor  elimination  and  refined 
dexterity  by  incorporating  wristed  instrumentation.  From 
250  robotic  cases  in  the  beginning  (2001),  the  number  has 
reached  73,000  in  2009  (86%  of  the  85,000  American  men 
who  had  prostate  cancer  surgery)  [3,4]. 

Initial  experiences  with  the  daVinci  surgical  system  have 
been  positive:  short  learning  curve,  limited  blood  loss,  less 
post-operative  pain,  favorable  complication  rates,  and  short 
hospital  stay  [3-10].  Despite  fewer  perioperative  compli¬ 
cations  and  shorter  hospital  stay,  a  recent  paper  found  pa¬ 
tients  were  three  times  more  likely  to  require  salvage  therapy 
[11] .  One  theoretical  disadvantage  with  regards  to  robotic 
surgery  is  the  lack  of  tactile  feedback.  In  open  RPs,  the  sur¬ 
geon  uses  his  fingers  to  feel  the  periphery  of  the  prostate 
gland  [12].  Without  tactile  feedback,  a  robotic  surgeon  is 
unable  to  appreciate  differences  in  tissue  texture  or  firm¬ 
ness  and  therefore  may  not  be  able  to  tailor  precisely  the 
extent  of  tissue  excision  around  the  prostate  gland  in  ef¬ 
forts  to  eradicate  all  cancerous  tissue.  Inadvertently  leaving 
residual  cancer  cells  behind,  called  a  positive  surgical  mar¬ 
gin  (PSM),  is  highly  associated  with  cancer  recurrence.  PSM 
rates  were  initially  higher  in  RALP  than  in  the  open  proce¬ 
dure,  but  they  have  been  shown  to  decrease  with  surgeon’s 
experience  and  improved  technique  [9,11]. 

As  manual  palpation  helps  guide  the  surgeon  in  the  open 
procedure,  an  equivalent  real-time  guiding  tool  is  needed 
for  robotic  prostatectomy.  Imaging  modalities  like  MRI  or 
CT  are  not  feasible  intraoperatively,  nor  do  they  possess  the 
sensitivity  or  specificity  for  accurate  detection  and  local¬ 
ization  of  prostate  cancer.  Transrectal  ultrasound  (TRUS) 
is  routinely  used  in  diagnosis,  in  conjunction  with  digital 
rectal  examination  (DRE)  and  biopsies  [13].  One  center 
used  TRUS  for  real-time  monitoring  and  guidance  during 
Laparoscopic  RP  and  reported  technical  feasibility  and  en¬ 
hanced  precision  by  decreased  PSM  rates  [14,15].  TRUS 
was  capable  of  imaging  a  substantial  percent  of  nonpal- 
pable  prostate  cancers.  The  authors  recognized  however, 
the  limitations  of  TRUS  guidance;  it  requires  considerable 
prior  expertise  and  tends  to  identify  primarily  hypoecho- 
ic  lesions,  which  were  just  47%  of  the  cancer  nodules  stud¬ 
ied  [15].  Today’s  prostate  cancer  patients  are  more  likely 
to  present  with  echogenic  or  isoechoic  lesions  because  ag¬ 
gressive  screening  techniques  lead  to  a  shift  toward  small¬ 
er,  early-stage  cancers  [16,17];  classic  B-mode  gray-scale  ul¬ 
trasound  alone  cannot  identify  these  lesions. 

Ultrasound  (US)  Elastography  (USE)  is  emerging  as  a  valu¬ 
able  tool  in  the  field  of  imaging.  Elastography  is  a  qualitative 
technique  based  on  the  principle  that  tissue  compression 


produces  strain  (displacement)  within  that  tissue;  strain  is 
smaller  in  harder,  stiffer  tissue  than  in  softer,  more  compli¬ 
ant  tissue  [18] .  Analyzing  the  ultrasound  raw  radio  frequen¬ 
cy  signal  results  in  a  strain  map,  commonly  called  elastogram , 
where  harder  tissue  is  darker  than  surrounding  soft  tissue. 
Cancers  tend  to  present  as  hard  lesions  due  to  increased  cel- 
lularity  [18] .  Echogenicity  and  stiffness  of  tissue  are  gener¬ 
ally  uncorrelated;  USE  can  identify  hypoechoic  lesions,  but 
also  echogenic  or  isoechoic  cancers  that  classic  gray-scale  ul¬ 
trasonography  cannot.  Elastography  through  the  transrec¬ 
tal  approach  has  already  been  proven  feasible  in  guiding  bi¬ 
opsies  of  the  prostate  [19-22].  Integrating  USE  technology 
with  a  laparoscopic  ultrasound  probe  will  give  robotic  and 
laparoscopic  surgeons  an  important  image-guidance  tool, 
which  until  this  point  does  not  exist  [23-25] . 

This  paper  describes  two  experiments  and  results  of  an  on¬ 
going  study  evaluating  the  diagnostic  accuracy  and  efficacy 
of  using  USE  to  identify  the  cancerous  nodules  in  the  pros¬ 
tate  gland.  The  aim  of  the  first  study  was  to  compare  the 
ability  of  subjects  to  detect  hard  tissue  (tumor  surrogates) 
in  synthetic  and  ex-vivo  phantoms.  We  attempted  to  mimic 
an  OR  setting  of  open  vs.  robotic  procedures,  by  asking  the 
subjects  to  identify  properties  of  the  tissue  using  manual  pal¬ 
pation  in  one  arm,  versus  using  ultrasound  elastograms  in 
the  other  arm.  The  elastograms  were  obtained  with  a  lapa¬ 
roscopic  ultrasound  probe.  In  the  second  study,  human  ex- 
vivo  prostatectomy  specimens  where  used  to  assess  the  ac¬ 
curacy  of  USE  in  the  identification  and  characterization  of 
hard  cancerous  nodules.  We  compared  the  elastogram  re¬ 
sults  with  histopathology  maps  (the  gold  standard)  and  also 
to  pre-  and  post-operative  MR  scans  of  the  prostate  gland 
in  order  to  assess  and  co-localize  anatomically  USE  with  the 
reference  histopathology  and  MR  scans. 

Material  and  Methods 


Comparative  study  for  USE  vs.  manual  palpation  in  tumor 
detection 

Institutional  Review  Board  approval  was  obtained  for  the 
comparative  study.  We  recruited  N=25  local  students  to  ass¬ 
es  human  ability  to  feel  hard  lesions  through  palpation,  ver¬ 
sus  elastography’s  ability  to  distinguish  the  same  lesions.  Our 
decision  to  use  local  students  instead  of  seasoned  surgeons 
stemmed  from  the  rationale  that  all  humans  are  born  with 
the  sense  of  touch  and  thus  have  the  innate  sensory  ability  to 
palpate;  we  were  also  able  to  recruit  more  subjects  in  order 
to  assess  inter-observer  variability.  Participants  were  asked  to 
identify  lesions  present  in  both  synthetic  and  ex-vivo  phan¬ 
toms.  The  subjects  evaluated  the  phantoms  using  manual 
palpation  and  ultrasound  based  elastograms.  The  hypothe¬ 
sis  of  the  study  was  that  subjects  could  identify  lesions  easi¬ 
er  on  the  elastograms  versus  using  manual  palpation.  Seven 
synthetic  and  four  ex-vivo  tissue  phantoms  were  created.  The 
phantoms  mimicked  the  mechanical  properties  of  prostate 
tissue  and  the  acoustic  scattering  properties  of  human  tis¬ 
sue.  Synthetic  phantoms  exhibited  deeper  spherical  hard  le¬ 
sions,  consistent  with  deeper  prostatic  cancerous  nodules, 
while  Ex-vivo  phantoms  had  superficial,  free-form  ablated  le¬ 
sions,  more  consistent  with  extra-capsular  cancer  extension. 

Synthetic  phantoms  (3x2x2  inches)  were  made  from  Liquid 
Plastic  (M-F  Manufacturing  Co.,  Inc.,  Haltom  City,  TX)  and 
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Figure  1 .  Synthetic  phantoms:  (A)  lesions  are 

visible  from  the  side  (pink)  and  the  top 
is  opaque  (dark  blue);  (B,C)  inside  the 
grid  calibration  container;  (D)  X-ray  of  ex- 
vivo  chicken  phantom.  Ablation  probe, 
tines  and  the  grid  calibration  container 
are  visible. 


Figure  2.  Laparoscopic  ultrasound  probe;  close-up  view  of  the  probe's 
head  (insert).  Prototype  courtesy  of  Intuitive  Surgical. 


glass  micro-beads.  The  micro-beads  were  used  as  a  scatter¬ 
ing  material.  They  were  added  to  the  plastic  mix  in  an  1% 
concentration  to  mimic  the  acoustic  scattering  properties 
of  human  tissue;  both  the  lesions  and  the  rest  of  the  phan¬ 
tom  appeared  isoechoic  under  B-mode  ultrasound.  Lesions 
were  created  by  varying  the  mixing  ratios  of  liquid  plastic 
types;  the  ratio  between  4116S  Plastic  Softener  or  7116  Plastic 
Hardener  and  8116  Super-Soft  Plastic  determined  the  final 
density  and  the  elastic  modulus  of  the  lesions  and  the  back¬ 
ground  [26] .  Each  phantom  contained  0-3  harder  spheri¬ 
cal  lesions  (1-2  cm  diameter)  colored  pink  for  ground  truth 
identification  (Figure  1  A) .  The  exposed  top  surface  was  col¬ 
ored  opaque  blue,  to  prevent  the  subjects  from  visually  iden¬ 
tifying  the  lesions  (Figure  1A) .  The  synthetic  phantoms  were 
sliced  and  sectioned  at  study  end,  following  axial  planes  par¬ 
allel  with  the  ultrasound  scanning  plane.  The  depth  of  each 
lesion  was  measured  as  the  distance  between  the  surface  of 
the  phantom  and  the  top  of  the  lesion  itself.  The  final  depth 
of  lesions  for  these  7  (seven)  phantoms  was  measured  to  be 
between  7  and  25  mm. 

Ex-vivo  phantoms  were  constructed  from  raw  chicken  breast 
tissue.  Hard  lesions  of  various  diameters  were  created  using 
radio  frequency  (RF)  ablation,  at  an  average  temperature  of 
95  degrees  Fahrenheit  for  20  minutes.  This  formed  a  hard 
spherical  lesion  at  a  depth  of  1-6  mm  below  the  surface, 


which  allowed  for  possible  palpation  but  not  the  visual  lo¬ 
calization  of  lesions.  Before  ablation,  each  tissue  was  placed 
in  a  small  plastic  container  and  surrounded  by  150  Bloom 
porcine  gelatin  (Bloom  represents  an  unit  of  measure  for  ri¬ 
gidity  of  gelatin) .  X-ray  axial  scans  (projection  plane  parallel 
with  the  ultrasound  scanning  plane)  were  used  to  localize  and 
measure  the  lesions  (Figure  ID).  The  tissue  phantoms  were 
sliced  on  the  same  axial  plane  at  study  end  to  determine  the 
depth  and  extent  of  the  ablated  areas.  The  depth  of  each  le¬ 
sion  was  measured  as  the  distance  between  the  surface  of  the 
phantom  and  the  top  of  the  lesion  itself.  For  ex-vivo  lesions, 
the  final  depth  of  lesions  was  measured  to  be  less  than  7  mm. 

Hardware  and  software  specification 

For  the  comparative  study,  a  laparoscopic  ultrasound  probe 
was  used,  fitted  with  a  transducer  (Gore  Tetrad,  Englewood, 
CO)  with  a  center  frequency  of  7.5  MHz,  and  128  elements 
(Figure  2)  [23-25].  Ultrasound  raw  radio-frequency  data 
was  acquired  using  an  Ultrasonix  US  scanner  (Ultrasonix 
Medical  Corporation,  Richmond,  BC,  Canada) .  Due  to  the 
relative  inexperience  of  our  subjects,  it  was  not  possible  to 
have  the  subjects  perform  real-time  elastography.  Thus,  to 
maintain  consistent  image  quality  and  to  minimize  user  de¬ 
pendence,  elastography  images  were  obtained  in  a  standard¬ 
ized  manner  by  one  of  our  researchers,  prior  to  the  evalua¬ 
tion.  Elastograms  were  generated  using  the  corresponding 
radio  frequency  data  and  our  dynamic  programming  (DP) 
elastography  algorithm  [27,28]. 

During  the  study,  each  subject  reviewed  3-4  phantoms,  each 
placed  in  a  self-designed  calibration  container,  which  con¬ 
sisted  of  a  5-by-5  grid,  0.5  inches  apart,  labeled  with  num¬ 
bers  and  letters  along  the  two  axis  (Figure  IB,  C).  Subjects 
were  asked  to  identify  by  manual  palpation  the  location 
based  on  the  provided  grid  (i.e.  B4  or  A3),  and  also  the 
diameter  and  depth  of  each  lesion  using  0.5  inches  as  the 
unit  of  reference.  For  the  USE  arm  of  the  study,  the  sub¬ 
jects  first  underwent  an  USE  training  session,  where  they 
were  explained  the  concept  and  shown  sample  elastograms. 
They  were  then  presented  with  3-4  elastograms  of  the  phan¬ 
toms  and  they  were  asked  to  provide  the  presence,  size  and 
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Comparison  of  root  mean  squared  error  of  estimation  of  lesion  depth 
Ultrasound  elastography  vs.  manual  palpation 


Manual  palpation 


Ultrasound  elastography 


Manual  palpation 


Ultrasound  elastography 


Figure  3.  Synthetic  phantom  study  results;  Box-and-whisker  plots  for  lesion  diameter  (A)  and  depth  from  surface  (B). 


depth  of  lesions  given  the  scale  of  the  images.  The  order  in 
which  the  subjects  completed  the  USE  and  manual  palpa¬ 
tion  tasks  was  randomized. 

Accuracy  was  determined  descriptively  using  box  and  whis¬ 
kers  plots  (Figure  3A,B),  sensitivity  and  specificity  calcula¬ 
tions,  and  root  mean  squared  error  of  estimation  (RMSE)  ob¬ 
tained  from  subtracting  the  estimated  value  of  the  measured 
parameter  (diameter,  depth)  from  the  ground  truth  value 
determined  from  direct  measurement.  STATA  9  (StataCorp 
LP,  College  Station,  TX)  was  used  to  perform  the  statisti¬ 
cal  analysis,  which  consisted  of  Student’s  t-test  for  compar¬ 
ison  between  the  means  of  the  RMSE  of  both  diameter  and 
depth  as  estimated  via  manual  palpation  versus  USE.  The 
p-values  reported  were  generated  by  t-test  calculations  as¬ 
suming  unequal  variances  for  a  two-tailed  test  where  p=0.05. 

Ex-vivo  human  prostate  study  for  tumor  detection 


Right 


Urethra 


Left 


Prostate  cancer  patients,  candidates  for  prostatectomy,  were 
prospectively  enrolled  in  our  study,  following  an  informed 
consent  approved  by  the  Institutional  Review  Board.  The  ob¬ 
jective  of  the  study  was  to  evaluate  the  efficacy  of  using  elas¬ 
tography  to  identify  and  precisely  localize  hard  nodules  such 
as  seen  with  prostate  cancer  just  beneath  the  surface  of  the 
prostate  gland  in  the  peripheral  zone.  In  this  area,  cancer¬ 
ous  lesions  are  at  most  risk  of  invasion  beyond  the  confines 
of  the  prostate  gland  and  also  more  likely  to  be  cut  across  by 
a  well  meaning  surgeon.  We  recruited  patients  who  under¬ 
went  both  open  and  assisted  prostatectomies  given  that  the 
process  of  removing  the  gland  was  not  a  focus  of  our  study. 
Patients  underwent  multiple  radiological  procedures.  1)  Pre¬ 
operative  3  Tesla  MRI  of  the  pelvis  was  performed  right  be¬ 
fore  the  surgery  procedure.  2)  Post-operative  ultra  high-reso¬ 
lution  MRI  at  9.4  Tesla  was  performed  on  the  excised  prostate 
specimen  to  correlate  the  results  to  in-vivo  pre-operative  im¬ 
aging.  3)  USE  was  then  performed  on  the  prostate  specimen 
by  an  experienced  radiologist  blinded  to  the  surgeon’s  find¬ 
ings  and  to  the  pre-operative  pathology  report.  The  collect¬ 
ed  radio-frequency  (RF)  data  was  used  offline  to  recreate 
classic  B-mode  grey-scale  images,  and  also  to  compute  elas- 
tograms  showing  the  stiffness  of  the  tissue  scanned  [27,28] . 

Pre-operative  and  post-operative  MR  scans  were  used  for  an¬ 
atomical  correlation  with  the  computed  elastograms.  For 


Figure  4.  Ultrasound  elastography  data  collection  process  using 
the  sextant  approach;  RF  data  was  acquired  in  axial 
planes  (1-6)  from  the  gland's  base  towards  the  apex.  For 
illustration  purposes,  a  lesion  is  outlined  in  the  left  mid 
section,  peripheral  zone  of  the  specimen,  similar  with  the 
case  of  specimen  #3. 


USE,  the  prostate  specimens  were  placed  in  prone  position 
on  a  surgical  table.  USE  scans  were  performed  in  a  system¬ 
atic  sextant  approach,  similar  to  that  used  for  image  guided 
biopsies.  RF  data  was  acquired  in  axial  planes  (from  gland’s 
base,  through  mid  gland,  to  apex)  on  the  left  and  right  side 
of  the  gland  (Figure  4) .  The  sextant  approach  was  neces¬ 
sary  to  ensure  that  the  scans  were  in  the  same  plane  with  the 
histopathology  diagrams  (axial)  which  constituted  the  gold 
standard  for  comparison.  USE  coronal  scans  from  the  left 
to  the  right  of  the  gland  were  also  performed;  these  scans 
were  in  alignment  with  the  MR  coronal  scans. 

Hardware  and  software  specification 

For  the  second  study,  USE  acquisition  was  conducted  using 
a  Siemens  Antares  US  scanner  (Siemens  Medical  Solutions 
USA,  Inc.  Ultrasound  Division,  Issaquah,  WA)  with  an 
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Figure  5.  Axial  section  of  prostate  specimen  #1  peripheral  zone.  Left  lateral  section  of  the  prostate's  base;  classic  ultrasound  B-mode  (A)  and 

elastogram  (B).  Hard  lesion  is  outlined,  arrows  point  to  adjacent  nodule.  (C)  Hematoxylin  &  eosin  stained  histological  section  of  prostate 
base.  The  tumor  (Gleason  score  3+5=8,  outlined  in  black)  extended  beyond  the  prostatic  capsule  in  this  section  and  invaded  the  left 
seminal  vesicle  (arrow). 


Table  1.  Table  summarizing  the  experimental  results  of  the  palpation 
study:  sensitivity  and  specificity. 


Manual 

palpation 

Ultrasound 

elastography 

Sensitivity 

66% 

84% 

Specificity 

67% 

71% 

Detection  rate  <20  mm  depth 

80% 

68% 

Detection  rate  >20  mm  depth 

0% 

66% 

Detection  rate 

80% 

84% 

ultrasound  research  interface  to  access  raw  RF  data.  Data 
was  acquired  by  manual  handling  using  a  Siemens  VF  10-5 
linear  array  for  prostate  specimens.  After  RF  data  collection, 
elastograms  were  obtained  using  the  dynamic  programming 
(DP)  elastography  algorithm  developed  in  our  lab  [27,28] . 

Each  prostate  specimen  underwent  routine  pathologic  pro¬ 
cessing  and  analysis.  Due  to  the  high  volume  of  prostatecto¬ 
mies  performed  at  our  institution,  the  routine  pathological 
process  does  not  result  in  a  whole  mount  mapping.  Instead, 
for  histopathological  evaluation  the  prostatectomy  speci¬ 
mens  were  initially  sliced  at  every  3-4  mm  from  apex  to 
base,  according  to  the  Stanford  protocol.  Each  slice  (6  to 
10  master  slices)  were  then  incorporated  in  a  paraffin  block 
and  sliced  at  5  p  meter  thickness.  The  slices  were  stained 
with  hematoxylin-eosin  and  were  then  analyzed  under  a  mi¬ 
croscope  by  a  pathologist  blinded  to  the  surgeon’s  findings 
and  also  to  the  elastography  results.  The  localization  and 
size  of  each  tumor  focus  were  documented  for  all  step  mas¬ 
ter  slices  on  axial  diagrams,  with  Gleason  score.  Large  mac¬ 
ro  photographs  were  reconstructed  in  several  specimens 
(Figure  5C).  All  data  collected  were  stored  in  the  database. 

N=10  target  areas  were  analyzed  from  N=6  patients  enrolled 
so  far  into  the  elastography  analysis.  Histological  findings 
served  as  the  gold  standard  in  determining  the  presence, 
location  and  size  of  any  prostatic  nodules,  malignant  and 


benign.  The  objective  of  our  study  was  then  to  compare  axi¬ 
al  elastograms  findings  with  the  histological  findings  record¬ 
ed  by  the  pathologist  (mapping  diagrams,  measurements 
and  nodule  characteristics  such  as  malignant  vs.  benign). 
Since  histopathology  diagrams  often  specified  just  the  max¬ 
imum  diameter  of  a  lesion,  coronal  elastograms  were  used 
to  better  establish  the  location  and  extent  of  the  identified 
lesions.  MRI  images  (both  axial  and  coronal  planes)  were 
aligned  to  the  elastograms  and  provided  help  with  their 
anatomical  co-registration  using  anatomical  details  such  as 
urethra  or  boundaries  of  peripheral  zone  vs.  central  gland. 

Results 

Comparative  study  for  tumor  detection 

Overall  sensitivity  and  specificity  results  are  summarized  in 
Table  1.  USE  showed  higher  accuracy  in  tumor  detection 
with  a  sensitivity  of  84%  and  specificity  of  71%,  compared 
to  a  sensitivity  of  66%  and  a  specificity  of  67%  for  manual 
palpation.  At  depths  greater  than  20  mm,  no  subject  was 
able  to  identify  a  lesion  by  manual  palpation.  66%  of  these 
lesions  were  correctly  identified  on  elastograms. 

Diameter  estimation  for  synthetic  phantoms  using  manual 
palpation  was  less  accurate  than  USE  (p=0.001)  with  a  root 
mean  squared  error  of  estimation  (RMSE)  of  4.81  for  man¬ 
ual  palpation  (95%  Cl  between  3.83  and  5.78),  versus  mean 
RMSE=3.02  for  USE  (95%  Cl  between  2.57  and  3.47).  For  ex- 
vivo  phantoms,  estimations  were  comparable  in  both  manu¬ 
al  palpation  and  USE,  at  a  RMSE  of  about  11.0  mm.  Depth 
estimation  for  synthetic  phantoms  was  statistically  higher  us¬ 
ing  manual  palpation  than  USE  (p=0.0001)  with  a  mean  val¬ 
ue  of  the  RMSE  of  8.81  for  manual  palpation  (95%  Cl  be¬ 
tween  6.24  and  11.39)  versus  a  mean  value  of  the  RMSE=3.02 
for  USE  (95%  Cl  between  2.42  and  3.63)  -  Figure  3.  For  ex- 
vivo  phantoms,  estimations  were  comparable  again  for  both 
manual  palpation  and  USE,  at  a  RMSE  of  about  1.0  mm. 

Ex-vivo  human  prostate  study 

Elastography  identified  N=10  lesions,  8  hard  nodules  in 
the  peripheral  zone,  1  hard  and  1  soft  nodule  in  the  cen¬ 
tral  gland  (Table  2).  Pathology  reports  showed  8  of  these 
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Table  2.  Prostate  specimen  data;  A  total  of  10  (ten)  elastography  lesions  were  identified  in  6  (six)  patients'specimens  (8  malignant  and  2  benign). 


# 

Location 

Gleason 

score 

Size  (cm) 

Elastography 

Pathology 

MRI 

1.1 

PZ  base 

3+5 
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PZ  -  peripheral  zone;  CG  -  central  gland. 
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Figure  6.  Coronal  section  of  prostate  specimen  #1 
at  the  level  of  the  central  gland.  Classic 
ultrasound  B-mode  (A)  and  elastogram 
(B).  9.4 Tesla  ex-vivo  (C)  and  3  Tesla  in- 
vivo  (D)  MRI  images  are  presented  in 
coronal  planes,  in  CCW  (counter  clock 
wise)  orientation  for  better  visualization 
of  the  correlation  between  USE  and  MRI 
of  the  specimen.  Benign  solid  (arrow) 
and  soft  (dashed  arrow)  nodules  and 
urethra  are  visible. 


lesions  as  malignant  and  2  as  benign.  Diameter  measure¬ 
ments  correlation  proved  difficult  because  of  the  inability 
to  perfectly  register  the  three  investigative  modalities.  USE 
and  MRI  measurements  were  within  on  average  2.05  mm  vs. 
2.25  mm  of  the  diameters  measured  by  pathology  (standard 
deviation  of  1.9  mm  for  USE  and  2.9  mm  for  MRI).  Size 
measurements  and  Gleason  score  are  reported  in  Table  2. 

Specimen  #1  presented  multiple  hard  and  soft  lesions,  lo¬ 
cated  in  the  central  gland  of  the  prostate  (Figure  6) .  The 
ex-vivo  T2-weighted  coronal  image  from  specimen  MRI 
obtained  at  9.4  Tesla  (Figure  6C)  -  here  in  counter  clock 
wise  orientation  for  better  visualization  of  the  correlation 


between  USE  and  MRI  of  the  specimen)  shows  detailed 
anatomy  of  the  heterogeneous  central  gland  with  a  solid 
benign  prostatic  hypertrophy  nodule  (BPH)  confirmed  by 
pathology.  Elastography  was  able  to  detect  this  solid  nod¬ 
ule  despite  the  heterogeneity  of  the  prostate  (Figure  6B)  - 
solid  arrow,  whereas  the  lesion  was  not  clearly  identified  by 
gray  scale  ultrasound.  Ex-vivo  T2-weighted  9.4  Tesla  coro¬ 
nal  image  from  specimen  MRI  also  shows  an  additional  soft 
cystic  BPH  nodule  (dashed  arrow).  Urethra  is  also  visible 
on  the  elastogram,  as  well  as  MRI  exam  (labeled  urethra). 

Axial  scans  of  the  same  specimens  were  compared  with  his- 
topathology  axial  cross-sections.  The  prostate,  submitted  for 
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Figure  7.  B-mode  image  (left)  and  elastogram  (right)  from  specimens  #  2-6.  Dark  regions  at  the  very  bottom  of  elastograms  represent  structures 
outside  the  prostate  tissue  (e.g.  operating  table).  The  border  of  the  prostatic  tissue  can  be  easily  noticed  as  a  highly  reflective  band  at  the 
bottom  of  B-mode  images. 


histological  processing  in  four  quadrant  sections  per  slice, 
was  digitally  realigned  to  reconstruct  a  full  histological  cross- 
section  (Figure  5C) .  Specimen  #1  was  found  with  a  tumor 
with  Gleason  score  of  8  at  the  prostate  base,  left  side  (out¬ 
lined)  .  Figure  5A  shows  an  ultrasound  B-mode  image  and 
an  elastogram  obtained  through  an  axial  plane  at  the  pros¬ 
tate’s  base  on  the  left  side.  The  same  tumor  was  identified 
by  elastography  (Figure  5B  -  dashed  contour)  but  is  not 
visible  on  grey-scale  ultrasound.  For  an  anatomical  corre¬ 
lation,  a  soft  -  cystic  nodule  anterior  to  cancer  can  be  seen 
on  B-mode  image  and  USE  (Figure  5A,B  -  arrows)  and  on 
histopathology  (Figure  5C  -  arrows). 

The  remaining  5  (five)  prostate  specimens  presented  with 
superficial  lesions  in  the  peripheral  zone  of  the  gland.  USE 
identified  multiple  hard  malignant  lesions  in  various  loca¬ 
tions,  from  the  base  to  the  apex  of  the  prostate  gland.  One 
can  notice  the  clear  delimitations  of  these  lesions  on  USE 
(Figure  7)  as  well  as  the  close  estimations  of  size  versus  pa¬ 
thology  and  MRI. 

Discussion 


In  surgical  procedures  where  manual  palpation  would  be 
helpful  but  not  possible  to  perform,  e.  g.  laparoscopic  robot¬ 
ic  surgery,  USE  can  offer  added  value  if  proven  to  be  accu¬ 
rate  in  detecting  pathologic  lesions.  Our  comparative  study 
showed  USE  to  be  superior  to  manual  palpation  in  general 
sensitivity  and  specificity,  and  also  in  identifying  deeper  le¬ 
sions.  Despite  of  the  inexperience  of  our  subjects  with  USE 
and  elastogram  evaluation,  USE  demonstrated  good  per¬ 
formance  in  detection  of  hard  lesions.  This  is  particularly 


important  as  surgeons  can  be  considered  inexperienced 
elastogram  readers  as  well.  Our  feasibility  study  showed  that 
USE  was  able  to  identify  both  hard  and  soft  lesions  in  the  ex- 
vivo  prostate  specimens,  located  in  the  deep  prostatic  central 
gland  and  in  the  peripheral  zone.  Histopathologic  findings 
validated  USE,  and  results  compared  favorably  with  the  in- 
vivo  pre-surgical  and  ex-vivo  post-surgical  MRI  scans.  In  the 
central  gland  of  the  prostate,  elastography  showed  excel¬ 
lent  detection  of  hard  and  soft  areas,  despite  the  complex¬ 
ity  of  the  central  gland.  Elastography  was  able  to  identify 
both  hard  and  soft  BPH  nodules  and  anatomical  landmarks 
like  the  urethra  (note  excellent  anatomical  correlation  to 
MR  scan  findings).  In  the  peripheral  zone  USE  identified 
multiple  hard  malignant  lesions,  from  the  base  to  the  apex 
of  the  prostate  gland.  These  preliminary  results  demon¬ 
strate  the  ability  of  USE  to  detect  hard  nodules  in  the  pros¬ 
tate  and  are  encouraging  in  the  pursuit  of  this  technology 
as  a  palpation  equivalent  imaging  tool  for  prostatectomy. 

USE  maps  tissue  elasticity  which  makes  it  an  ideal  imaging 
modality  to  serve  as  a  surrogate  and  possible  equivalence 
to  manual  palpation  in  identifying  hard  cancerous  tissue 
in  the  prostate  gland,  especially  in  the  peripheral  zone  but 
also  in  the  central  gland.  Real-time  intra-operative  imaging 
guidance  is  needed  for  identifying  the  presence  of  cancer 
within  the  prostate,  especially  near  the  capsule  where  tu¬ 
mor  can  invade  and  spread  outside  of  the  gland,  and  also 
for  studying  surrounding  structures.  If  diseased  hard  lymph 
nodes  could  be  detected,  then  lymphadenectomy  may  pro¬ 
vide  a  more  accurate  cancer  staging,  help  tailor  future  thera¬ 
py,  and  potentially  prevent  recurrence.  A  better  delineation 
of  the  bladder  neck  and  apex  during  dissection,  especially 
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when  prostate  cancer  is  located  at  the  apex  could  perhaps 
improve  patients’  outcome.  If  deemed  possible,  imaging  cav¬ 
ernous  nerves  (CNs)  located  along  the  immediate  surface 
of  the  prostate  gland  may  lead  to  their  preservation,  and 
thus  improved  preservation  of  potency  and  urinary  conti¬ 
nence  [29] .  Further  more,  the  development  of  the  elastog- 
raphy  technology  as  an  imaging  guiding  tool  during  prosta¬ 
tectomy  could  potentially  be  useful  in  the  open  procedures 
as  well,  where  the  manual  palpation  would  not  be  enough 
to  identify  deeper  lesions.  It  has  been  documented  in  the 
literature  that  prostate  carcinoma  originates  in  the  central 
gland  and  transitional  zone  in  up  to  30%  of  cases  [30,31] . 

Conclusions 


Our  initial  experience  showed  USE  was  able  to  reliably  iden¬ 
tify  hard  nodules  in  the  peripheral  zone  of  the  prostate  that 
were  prostate  cancers.  Additionally,  USE  showed  its  ability 
to  define  tissue  hardness  of  BPH  nodules  despite  the  un¬ 
derlying  tissue  complexity  in  the  central  gland.  Our  com¬ 
parative  study  demonstrated  USE  can  approach  the  effica¬ 
cy  of  manual  palpation  for  superficial  lesions  and  has  the 
potential  to  surpass  it  for  smaller,  deeper  lesions.  We  em¬ 
ployed  a  laparoscopic  ultrasound  probe  which  was  success¬ 
fully  used  and  tested  in  conjunction  with  elastography  al¬ 
gorithms.  Our  initial  experience  with  USE  encourages  us 
to  pursue  further  the  evaluation  of  this  technique.  Further 
testing  of  the  laparoscopic  probe  is  needed  in  a  real  lapa¬ 
roscopic  environment.  We  can  conclude  that  there  is  prom¬ 
ise  is  integrating  laparoscopic  ultrasound  elastography  as 
a  real-time,  in-vivo  imaging  tool  to  guide  surgeons  during 
robotic-assisted  prostatectomies. 

Acknowledgements 

We  thank  our  colleagues,  surgeon  Mohammed  Allaf  and 
Naima  Carter-Monroe,  and  also  Intuitive  Surgical  for  their 
help.  Ioana  Fleming  is  supported  by  the  Department  of 
Defense  Prostate  Cancer  Predoctoral  Fellowship. 

References: 


1.  Jemal  A,  Siegel  R,  XuJ,  Ward  E:  Cancer  statistics,  2010.  CA:  A  Cancer 
Journal  for  Clinicians.  2010;  60(5):  277-300 

2.  Bill-Axelson  A,  Holmberg  L,  Ruutu  M  et  al:  Radical  prostatectomy  versus 
watchful  waiting  in  early  prostate  cancer.  N  EnglJ  Med,  2011;  364(18): 
1708-17 

3.  Box  GN,  Ahlering  TE:  Robotic  radical  prostatectomy:  long-term  out¬ 
comes.  Curr  Opin  Urol,  2008;  18(2):  173-79 

4.  Pruthi  RS,  Wallen  EM:  Current  status  of  robotic  prostatectomy:  prom¬ 
ises  fulfilled.  J  Urol,  2009;  181(6):  2420-21 

5.  Nelson  B,  Kaufman  M,  Broughton  G  et  al:  Comparison  of  length  of  hos¬ 
pital  stay  between  radical  retropubic  prostatectomy  and  robotic  assist¬ 
ed  laparoscopic  prostatectomy.  J  Urol,  2007;  177(3):  929-31 

6.  Artibani  W,  Fracalanza  S,  Cavalleri  S  et  al:  Learning  curve  and  prelim¬ 
inary  experience  with  da  Vinci-assisted  laparoscopic  radical  prostatec¬ 
tomy.  Urol  Int,  2008;  80(3):  237-44 

7.  Colombo  J,  Jose  R,  Santos  B  et  al:  Robotic  assisted  radical  prostatecto¬ 
my:  surgical  techniques  and  outcomes.  International  BrazJ  Urol,  2007; 
33(6):  803-9 


8] .  Badani  KK,  Kaul  S,  Menon  M:  Evolution  of  robotic  radical  prostatecto¬ 
my:  assessment  after  2766  procedures.  Cancer,  2007;  110(9):  1951-58 

9.  Ficarra  V,  Cavalleri  S,  Novara  G  et  al:  Evidence  from  robot-assisted  lap¬ 
aroscopic  radical  prostatectomy:  a  systematic  review.  Eur  Urol,  2007; 
51(1):  45-55;  discussion  56 

10.  Berryhill  J  Roy,  Jhaveri  J,  Yadav  R  et  al:  Robotic  prostatectomy:  a  re¬ 
view  of  outcomes  compared  with  laparoscopic  and  open  approaches. 
Urology,  2008;  72(1):  15-23 

11.  HuJC,  Wang  Q,  Pashos  CL  et  al:  Utilization  and  outcomes  of  minimal¬ 
ly  invasive  radical  prostatectomy.  J  Clin  Oncol,  2008;  26(14):  2278-84 

12.  Walsh  PC,  Lepor  H,  Eggleston  JC:  Radical  prostatectomy  with  preser¬ 
vation  of  sexual  function:  anatomical  and  pathological  considerations. 
Prostate,  1983;  4(5) :  473-85 

13.  Salomon  G,  Kllerman  J,  Thederan  I  et  al:  Evaluation  of  prostate  can¬ 
cer  detection  with  ultrasound  real-time  elastography:  a  comparison  with 
step  section  pathological  analysis  after  radical  prostatectomy.  Eur  Urol, 
2008;  54(6):  1354-62 

14.  Ukimura  O,  Gill  IS,  Desai  MM  et  al:  Real-time  transrectal  ultrasonog¬ 
raphy  during  laparoscopic  radical  prostatectomy.  J  Urol,  2004;  172(1): 
112-18 

15.  Ukimura  O,  Magi-Galluzzi  C,  Gill  IS:  Real-time  transrectal  ultrasound 
guidance  during  laparoscopic  radical  prostatectomy:  impact  on  surgi¬ 
cal  margins.  J  Urol,  2006;  175(4):  1304-10 

16.  Coley  CM,  Barry  MJ,  Fleming  C,  Mulley  AG:  Early  detection  of  prostate 
cancer.  Part  I:  Prior  probability  and  effectiveness  of  tests.  Ann  Intern 
Med,  1997;  126(5):  394-406 

17.  Daehnert  WF,  Hamper  UM,  Eggleston  JC  et  al:  Prostatic  evaluation  by 
transrectal  sonography  with  histopathologic  correlation:  the  echope- 
nic  appearance  of  early  carcinoma.  Radiology,  1986;  158(1):  97-102 

18.  Ophir  J,  Alam  SK,  Garra  B  et  al:  Elastography:  ultrasonic  estimation 
and  imaging  of  the  elastic  properties  of  tissues.  Proc  Inst  Mech  Eng  H, 
1999;  213(3):  203-33 

19.  Cochlin  DL,  Ganatra  RH,  Griffiths  DFR:  Elastography  in  the  detection 
of  prostatic  cancer.  Clin  Radiol,  2002;  57(11):  1014-20 

20.  Koenig  K,  Scheipers  U,  Pesavento  A  et  al:  Initial  experiences  with  real¬ 
time  elastography  guided  biopsies  of  the  prostate.  J  Urol.  2005;  174(1): 
115-17 

21.  Sumura  M,  Shigeno  K,  Hyuga  T  et  al:  Initial  evaluation  of  prostate  can¬ 
cer  with  real-time  elastography  based  on  step-section  pathologic  anal¬ 
ysis  after  radical  prostatectomy:  a  preliminary  study.  Int  J  Urol,  2007; 
14(9):  811-16 

22.  Leven  J,  Burschka  D,  Kumar  R  et  al:  DaVinci  canvas:  a  telerobotic  surgi¬ 
cal  system  with  integrated,  robot-assisted,  laparoscopic  ultrasound  capa¬ 
bility.  Med  Image  Comput  Comput  Assist  Interv,  2005;  8(Pt  1):  811-18 

23.  Schneider  CM,  Dachs  GW  II,  Hasser  CJ  et  al:  Robot-assisted  laparoscop¬ 
ic  ultrasound.  In:  Proceedings  of  the  First  international  conference  on 
Information  processing  in  computer-assisted  interventions.  IPCAI’10. 
Berlin,  Heidelberg:  Springer-Verlag,  2010;  67-80 

24.  Schneider  CM,  Peng  PD,  Taylor  RH  et  al:  Robot-assisted  laparoscopic 
ultrasonography  for  hepatic  surgery.  Surgery,  2012;  151(5):  756-62 

25.  Billings  S,  Deshmukh  N,  Kang  HJ  et  al:  System  for  robot-assisted  real¬ 
time  laparoscopic  ultrasound  elastography.  In:  Proceedings  of  the  SPIE 
Medical  Imaging,  2012;  67 

26.  Mansy  HA,  Grahe  JR,  Sandler  RH:  Elastic  properties  of  synthetic  mate¬ 
rials  for  soft  tissue  modeling.  Phys  Med  Biol,  2008;  53(8):  2115-30 

27.  Rivaz  H,  Boctor  E,  Foroughi  P  et  al:  Ultrasound  elastography:  a  dynam¬ 
ic  programming  approach.  IEEE  Trans  Med  Imaging,  2008;  27(10): 
1373-77 

28.  Rivaz  H,  Boctor  EM,  Choti  MA,  Hager  GD:  Real-time  regularized  ultra¬ 
sound  elastography.  IEEE  Trans  Med  Imaging,  2011;  30(4):  928-45 

29.  Su  L,  Link  RE,  Bhayani  SB  et  al:  Nerve-sparing  laparoscopic  radical 
prostatectomy:  replicating  the  open  surgical  technique.  Urology,  2004; 
64(1):  123-27 

30.  McNeal  JE,  Redwine  EA,  Freiha  FS,  Stamey  TA:  Zonal  distribution  of 
prostatic  adenocarcinoma.  Correlation  with  histologic  pattern  and  di¬ 
rection  of  spread.  AmJ  Surg  Pathol,  1988;  12(12):  897-906 

31.  Cohen  iq,  Shannon  BA,  Phillips  M  et  al:  Central  zone  carcinoma  of  the 
prostate  gland:  a  distinct  tumor  type  with  poor  prognostic  features.  J 
Urol,  2008;  179(5):  1762-67;  discussion  1767 


CR642 


Electronic  PDF  security  powered  by  ISL-science.com 


Surgical  Technique 


Robot-assisted  laparoscopic 
ultrasonography  for  hepatic  surgery 

Caitlin  M.  Schneider,  BS,a  Peter  D.  Peng,  MD,b  Russell  H.  Taylor,  PhD,a  Gregory  W.  Dachs  II,  MS,C 
Christopher  J.  Hasser,  PhD,c  Simon  P.  DiMaio,  PhD,c  and  Michael  A.  Choti,  MD,  MBA,  FACS,a 

Baltimore,  MD,  and  Sunnyvale,  CA 

Introduction.  This  study  describes  and  evaluates  a  novel,  robot-assisted  laparoscopic  ultrasonographic 
device  for  hepatic  surgery.  Laparoscopic  liver  surgery  is  being  performed  with  increasing  frequency.  One 
major  drawback  of  this  approach  is  the  limited  capability  of  intraoperative  ultrasonography  (IOUS) 
using  standard  laparoscopic  devices.  Robotic  surgery  systems  offer  the  opportunity  to  develop  new  tools  to 
improve  techniques  in  minimally  invasive  surgery.  This  study  evaluates  a  new  integrated  ultrasonog¬ 
raphy  (US)  device  with  the  da  Vinci  Surgical  System  for  laparoscopic  visualization,  comparing  it  with 
conventional  handheld  laparoscopic  IOUS  for  performing  key  tasks  in  hepatic  surgery. 

Methods.  A  prototype  laparoscopic  IOUS  instrument  was  developed  for  the  da  Vinci  Surgical  System  and 
compared  with  a  conventional  laparoscopic  US  device  in  simulation  tasks:  (1)  In  vivo  porcine  hepatic 
visualization  and  probe  manipulation,  (2)  lesion  detection  accuracy,  and  (3)  biopsy  precision.  Usability 
was  queried  by  poststudy  questionnaire. 

Results.  The  robotic  US  proved  better  than  conventional  laparoscopic  US  in  liver  surface  exploration 
(85%  success  vs  73%;  P  =  .030)  and  tool  manipulation  (79%  vs  57%;  P  =  .028),  whereas 
no  difference  was  detected  in  lesion  identification  (63  vs  58;  P  =  .41)  and  needle  biopsy  tasks  (57  vs  48; 
P  =  .11).  Subjects  found  the  robotic  US  to  facilitate  better  probe  positioning  (80%  ),  decrease  fatigue 
(90%),  and  be  more  useful  overall  (90%)  on  the  post-task  questionnaire. 

Conclusion.  We  found  this  robot-assisted  IOUS  system  to  be  practical  and  useful  in  the  performance  of 
important  tasks  required  for  hepatic  surgery,  outperforming  free-hand  laparoscopic  IOUS  for  certain 
tasks,  and  was  more  subjectively  usable  to  the  surgeon.  Systems  such  as  this  may  expand  the  use  of  robotic 
surgery  for  complex  operative  procedures  requiring  IOUS.  (Surgery  2012;151:756-62.) 


From  the  Department  of  Surgery,61  The  Johns  Hopkins  Hospital;  the  Department  of  Computer  Science, b  The 
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Over  the  last  2  decades,  advances  in  the  operative 
management  of  liver  malignancies  have  resulted  in 
increased  patient  survival.  Complete  resection  of 
hepatocellular  carcinoma  or  colorectal  liver  metas¬ 
tasis  can  now  achieve  overall  5-year  survival  of 
>50%. 1-3  Laparoscopic  liver  resection  is  being 
utilized  increasingly,  with  approximately  3000 
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published  cases  worldwide.4  As  observed  with  lapa¬ 
roscopic  approaches  in  other  fields  of  surgery,  the 
advantages  include  less  postoperative  pain,  smaller 
incision  scars,  and  lesser  durations  of  hospitaliza¬ 
tion.5"9  Familiarity  with  both  hepatobiliary  surgery 
and  advanced  laparoscopy  are  necessary  for  suc¬ 
cessful  laparoscopic  liver  surgery.  Several  recent  re¬ 
ports  of  robotic  liver  surgery  have  introduced  the 
potential  advantages  of  the  use  of  robotic  assis¬ 
tance  in  minimally  invasive  liver  surgery.10,11 

The  da  Vinci  Surgical  System  (Intuitive  Surgical, 
Mountain  View,  CA)  combines  high  dexterity  tele- 
robotic  control  of  advanced  laparoscopic  instru¬ 
ments  with  high-fidelity,  3-dimensional  (3D) 
visualization  to  give  surgeons  the  ability  to  perform 
complex  operations  through  a  minimally  invasive 
approach.  Experiences  in  urology  and  cardiac 
surgery  have  already  demonstrated  outcomes 
equivalent  or  better  than  those  of  open  surgery 


756  SURGERY 


Surgery 

Volume  151,  Number  5 


Schneider  et  al  757 


Fig  1.  (A)  An  Aloka  UST-5536-7.5  hand-held  laparoscopic  US  probe  (top)  and  the  da  Vinci  US  instrument  prototype 
(bottom).  (B)  A  close-up  of  the  Aloka  (top)  and  da  Vinci  (bottom)  transducers  and  wrists.  (C)  The  US  instrument  manip¬ 
ulated  by  a  da  Vinci  robotic  manipulator. 


and  similar  to  laparoscopic  surgery.12,13  Improved 
ergonomic  dexterity  and  3D  magnified  visualiza¬ 
tion  offer  advantages  over  traditional  laparoscopy 
in  terms  of  usability.  Robotic  assistance  may  ex¬ 
pand  the  use  of  minimally  invasive  techniques  in 
hepatobiliary  surgery  and  the  development  of  ad¬ 
ditional  robotic  resection,  ablation,  and  new  imag¬ 
ing  tools  will  help  to  potentiate  its  adoption. 

Both  open  and  laparoscopic  liver  surgery  de¬ 
pends  greatly  on  intraoperative  ultrasonography 
(IOUS)  for  the  evaluation  of  vascular  and  biliary 
anatomy,  identification  of  known  and  occult  intra- 
hepatic  lesions,  and  operative  planning.1416  Al¬ 
though  laparoscopic  handheld  ultrasonography 
(US)  can  be  used  currently  in  robotic  surgery,  we 
have  found  it  to  be  a  cumbersome  approach. 
The  introduction  of  a  practical  integrated  US  in¬ 
strument  may  facilitate  the  wider  adoption  of 
robotic  liver  surgery. 

In  this  study,  we  describe  the  development  of  a 
high  dexterity  robotic  laparoscopic  US  tool  (RLUS) 
for  the  da  Vinci  surgical  robot  with  integration  into 
our  previously  described  open-source  research  en¬ 
vironment  of  imaging  software.17,18  Our  central 
goal  focuses  on  the  need  for  an  easily  usable,  inte¬ 
grated,  surgical  US  tool  in  the  robotic  environment 
that  expands  the  application  of  robotic  surgery  to 
hepatobiliary  operations.  We  report  the  general 
specifications  and  capabilities  of  RLUS  and  detail 
comparative  experiments  comparing  the  RLUS  sys¬ 
tem  with  hand-held  laparoscopic  US  in  tasks  that  are 
encountered  in  liver  surgery. 

METHODS 

US  instrument  design.  A  prototype  da  Vinci 
laparoscopic  US  instrument  was  developed  based 


on  the  5-mm  EndoWrist  instrument  architecture  but 
scaled  to  a  diameter  of  10  mm  to  accommodate  an 
off-the-shelf  linear  laparoscopic  transducer  (Gore 
Tetrad,  Englewood,  CO).  The  5-mm  wrist  is  based 
on  a  cable-driven,  multilink,  snake  architecture 
that— when  scaled  to  10  mm— is  able  to  accommo¬ 
date  the  coaxial  cable  bundle  that  is  routed  through 
the  center  of  the  instrument  shaft  from  the  trans¬ 
ducer  to  the  system  cable  interface  at  the  rear  of  the 
instrument  (Fig  1). 

The  linear  transducer  contains  128  elements, 
has  a  total  array  length  of  46  mm,  and  operates  at  a 
center  frequency  of  7.5  MHz.  In  terms  of  geometry 
and  imaging  performance,  the  RLUS  instrument  is 
similar  to  standard  hand-held  laparoscopic  probes 
that  are  in  use  today,  such  as  the  Aloka  UST-5536- 
7.5  (Aloka  America,  Wallingford,  CT).  The  articu¬ 
lated  wrist  provides  a  6-degree-of-freedom  control 
of  the  probe,  from  the  master  tool  manipulators  of 
the  surgical  console. 

Image  visualization.  An  open-source  software 
framework  has  been  used  to  display  US  images, 
probe  status,  and  guidance  information  in  the  stereo 
display  of  the  robotic  surgical  console.  B-Mode  US 
images  can  be  displayed  in  a  variety  of  ways: 

•  A  split  screen  display  mode  in  which  the  surgeon  sees 
the  endoscopic  and  US  views  side  by  side  (Fig  2,  A) .  To 
standardize  the  experiment,  this  mode  was  prese¬ 
lected  for  all  subsequent  usability  experiments. 

•  A  picture-in-picture  display  mode  that  insets  the  US 
image  into  the  endoscopic  view.  In  this  configuration, 
the  surgeon  is  able  to  select  the  position  and  size  of 
the  inset  image  by  manipulating  the  master  tool  ma¬ 
nipulators  within  the  console— this  is  a  user  interface 
feature  that  is  provided  by  a  3D  user  interface  module 
implemented  within  the  software  library. 


758  Schneider  et  al 


Surgery 
May  2012 


Fig  2.  (A)  A  split-screen  display  that  shows  the  endoscopic  view  (top)  and  the  US  image  ( bottom )  adjacent  to  one  another 
in  the  surgical  console.  ( B )  The  US  “flashlight”  overlay.  (C)  The  display  overlays  for  visualizing  probe  orientation 
(probe  widget  tool)  and  for  mapping  probe  waypoints  (mapping  tool) . 


•  A  “flashlight”  display  mode  in  which  the  US  image  is 
overlaid  onto  a  3D  representation  of  the  imaging 
plane  in  the  stereo  view  of  the  console.  The  effect  of 
this  mode  is  to  display  the  US  image  in  the  plane  in 
which  it  is  physically  acquired  by  the  transducer, 
such  that  the  image  is  co-located  with  the  view  of  the 
tissue  that  is  being  imaged  in  the  surgical  field.  Issues 
of  automatic  calibration  and  image-probe  registration 
were  addressed  in  our  prior  work  with  a  nonarticu- 
lated  probe17  (Fig  2,  B). 

In  addition  to  the  image  overlay,  the  system 
displays  a  graphic  representation  of  the  probe, 
imaging  plane,  and  wrist  configuration  at  the 
lower  margin  of  the  endoscopic  view.  This 
graphic  widget  provides  the  user  with  cues  for 
orienting  the  imaging  plane  of  the  probe,  as  well 
as  for  avoiding  wrist  range  of  motion  limits, 
particularly  when  the  US  probe  fills  the  field  of 
view  of  the  endoscope  and  the  wrist  is  not  visible 
(Fig  2,  C). 

Usability  experiments.  A  usability  study  was 
conducted  to  evaluate  the  performance  of  the 
RLUS  and  compare  it  with  a  conventional  hand¬ 
held  laparoscopic  device.  The  study  was  approved 
through  The  Johns  Hopkins  University  Institu¬ 
tional  Review  Board,  and  there  were  no  protocol 
violations.  Board-certified/ eligible  surgeons  with 
experience  in  laparoscopy  and  IOUS  met  enroll¬ 
ment  criteria.  A  total  of  10  subjects  participated  in 
the  study:  7  completing  the  full  protocol  and  3 
completing  only  lesion-finding  tasks. 


Table  I.  Methods:  task  categories  and  score 
weighting 

% 

Imaging  and  anatomic  structure  identification 


Liver  surface  exploration  27 

Anatomy  identification  55 

Tool  manipulation  9 

Image  quality  9 

Lesion  detection 

Number  of  lesions  found  33 

Accuracy  of  volumetric  measurement  33 

Self-assessed  confidence  in  lesion  finding  33 

Needle  biopsy  guidance 

Number  of  positive  biopsies  33 

Number  of  access  attempts  33 

Total  time  needed  33 


Hepatic  surgery  was  the  focusing  application 
and  specific  tasks  were  based  on  surgical  relevance 
and  difficulty  with  traditional  laparoscopic  tech¬ 
niques.  As  categorized  in  Table  I,  the  tasks 
included:  (1)  general  quality  of  liver  surface  acces¬ 
sibility,  volume  imaging,  and  intrahepatic  anatomic 
structure  identification  (n  =  7);  (2)  lesion  detection 
(n  =  10);  and  (3)  needle  biopsy  guidance  (n  =  8). 
The  time  to  complete  each  task  was  recorded,  in  ad¬ 
dition  to  other  specific  measures  related  to  the  suc¬ 
cessful  completion.  A  short  period  of  practice  was 
allowed  for  each  task.  The  task  was  explained  in  de¬ 
tail  before  beginning  and  all  questions  from  the 
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subjects  were  answered  at  this  time.  Each  task  was 
completed  with  both  the  RLUS  and  the  handheld 
laparoscopic  US.  The  order  of  instrument  usage 
was  randomized.  A  questionnaire  assessing  instru¬ 
ment  usability  was  administered  immediately  after 
completion  of  the  tasks. 

Task  1:  Imaging  and  anatomic  structure  identifica¬ 
tion :  Subjects  were  asked  to  manipulate  the  US 
probe  over  the  entire  exposed  anterior  surface  of 
in  vivo  porcine  liver  while  imaging  as  much  of  the 
liver  volume  as  possible.  They  were  then  asked  to 
identify  and  image-capture  specific  hepatic  struc¬ 
tures  in  2  views:  The  portal  vein  confluence, 
hepatic  vein  confluence,  inferior  vena  cava,  and 
gallbladder.  Still  images  and  video  recordings  were 
captured  and  subsequently  scored  blinded  by  ex¬ 
pert  observers.  Anatomic  identification  accounted 
for  24  out  of  44  possible  points  in  this  task,  with 
greater  weight  owing  to  complexity  of  subtask. 
Liver  surface  exploration  was  scored  out  of  12 
points,  and  both  tool  manipulation  and  image 
quality  were  scored  out  of  4  points. 

Task  2:  Lesion  detection :  Ex  vivo  study  with  phan¬ 
toms  were  constructed  of  PVC  plastic  as  described 
previously.19  Briefly,  both  hypoechoic  and  hyper- 
echoic  lesions  were  constructed  with  glass  micro¬ 
spheres.  Varying  in  echogenicity,  depth,  and  size, 
1-8  lesions  were  placed  in  each  phantom.  Subjects 
were  provided  a  phantom  without  knowledge  of 
the  number  of  lesions  contained.  They  were  then 
asked  to  identify  and  measure  all  lesions  as  accu¬ 
rately  and  rapidly  as  possible.  The  score  for  this 
task  was  compiled  from  (1)  the  number  of  lesions 
found  within  each  phantom,  (2)  the  accuracy  in 
lesion  volumetric  measurements,  and  (3)  the  self- 
assessed  confidence  in  finding  all  lesions.  Each  cat¬ 
egory  was  scored  out  of  12  points  for  a  total  task 
score  of  36  possible  points. 

Task  3:  Needle  biopsy  guidance*.  This  task  simulated 
clinical  US-guided  needle-based  tasks,  such  as 
tumor  biopsy  and  ablation.  Phantoms  were  devel¬ 
oped  from  ex  vivo  bovine  liver  and  target  1-cm 
lesions  created  using  intraparenchymally  injected 
dental  alginate  polymer.20  The  liver  phantom  was 
then  placed  in  a  visually  shielded  torso  model.  Sub¬ 
jects  were  asked  to  guide  and  perform  a  core  bi¬ 
opsy  of  the  target  lesion  using  a  spring-loaded 
biopsy  gun,  conducted  either  using  robotic  or 
free-hand  laparoscopic  assistance.  This  task  was 
scored  as  successful  on  recovery  and  visual  inspec¬ 
tion  of  white  alginate  material  within  the  biopsy 
core.  Overall  score  was  compiled  from  the  sum 
of  3  categories:  (1)  number  of  positive  biopsies, 
(2)  number  of  access  attempts,  and  (3)  total  time 
necessary  to  acquire  biopsies.  Each  category  was 


scored  out  of  12  for  a  total  possible  score  of  36 
for  this  task. 

Statistical  analysis.  Each  score  was  calculated, 
and  the  subtasks  were  compiled  into  task  scores  as 
described  previously.  The  weighting  for  the  sub¬ 
tasks  reflected  the  importance  and  measurement 
accuracy  in  each  grouping.  The  mean  and  stan¬ 
dard  deviations  were  calculated  for  each  task. 
Paired  t  test  comparison  was  performed  for  each 
sub  task  and  task  group. 

Questionnaire.  Study  subjects  were  adminis¬ 
tered  a  survey  immediately  after  completion  of 
the  US  simulation  tasks.  This  questionnaire  was 
composed  of  14  questions  comparing  robotic  ver¬ 
sus  conventional  handheld  laparoscopic  US  de¬ 
vices  for  usability,  ergonomics,  and  effectiveness. 
The  survey  aimed  to  address  the  subjective  expe¬ 
rience  of  the  subject  when  using  the  2  devices. 

RESULTS 

US  task  completion  experiment.  The  findings 
from  the  task  performance  between  RLUS  and 
conventional  handheld  laparoscopic  IOUS  are 
summarized  in  Table  II. 

Task  1:  For  imaging  and  anatomic  identification 
tasks,  the  RLUS  scored  greater  in  several  sub  task 
categories  including  liver  surface  exploration 
(85%  vs  73%;  P  =  .030)  and  tool  manipulation 
(79%  vs  57%;  P=  .028).  The  RLUS  had  similar  abil¬ 
ity  anatomic  identification  (76%  vs  62%;  P=  .12). 
There  was  no  difference  in  time  for  surface  explo¬ 
ration  between  the  2  groups. 

Task  2:  The  RLUS  demonstrated  no  difference 
in  scoring  for  the  remainder  of  the  tasks.  For 
lesion  detection  tasks,  the  RLUS  and  handheld 
laparoscopic  US  scored  similarly  (72%  vs  71%NS). 
There  was  no  difference  in  questionnaire  assessed 
confidence  in  lesion  identification  (73%  vs  70%). 
The  time  for  completing  lesion  identification  was 
greater  when  using  the  RLUS  (mean,  10.7  vs  7.9 
minutes;  P  =  .008);  however,  there  was  no  differ¬ 
ence  in  the  time  for  lesion  volume  assessment 
(6.1  vs  7.1  minutes;  P  =  .11). 

Task  3\  For  needle  biopsy  guidance  tasks,  the 
RLUS  did  not  demonstrate  significant  perfor¬ 
mance  differences.  These  tasks  included  confi¬ 
dence  in  positive  biopsy  (64%  vs  60%),  positive 
biopsy  (50%  vs  42%),  and  number  of  punctures 
needed  for  successful  biopsy  (2.7  vs  4.6;  P=  .11). 
There  was  no  difference  in  time  to  completion 
for  biopsy  tasks  between  RLUS  and  handheld 
laparoscopic  US. 

As  depicted  in  Fig  3,  the  RLUS  was  comparable 
to  the  handheld  laparoscopic  US  in  surface 
exploration,  tool  manipulation,  and  anatomic 
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Table  II.  Results:  task  scoring,  averages,  standard  deviations  and  statistical  comparison 


Task 

da  Vinci  mean  ±  a  [%] 

Handheld  mean  ±  <r  [%] 

P  -value 

Task  1:  imaging  and  anatomic  structure  identification 

Liver  surface  exploration 

85  ±  9 

73  ±  13 

.03 

Anatomy  identification 

76  ±  14 

62  ±  11 

NS 

Tool  manipulation 

79  ±  9 

57  ±  19 

.028 

Image  quality 

93  ±  12 

96  ±  10 

NS 

Combined  imaging  task  score 

80  ±  7 

68  ±  8 

.026 

Task  2:  lesion  detection 

Lesions  found 

72  ±  16 

71  ±  11 

NS 

Lesion  volume  error 

28  ±  11 

32  ±  14 

NS 

Confidence  in  lesion  identification 

73  ±  15 

70  ±  21 

NS 

Overall  lesion  task  score 

63  ±  16 

58  ±  18 

NS 

Task  3:  needle  biopsy  guidance 

Confidence  in  positive  biopsy 

64  ±  21 

59  ±  19 

NS 

Positive  biopsy 

50  ±  36 

42  ±  30 

NS 

Average  number  of  punctures 

2.7  ±  2.1 

4.6  ±  3.8 

NS 

Overall  biopsy  task  score 

57  ±  20 

48  ±  20 

NS 

NS,  Not  significant. 


identification,  and  required  a  similar  number  of 
requisite  passes  for  successful  biopsy. 

Questionnaire.  All  10  subjects  completed  the 
questionnaire  after  the  US  task  experiments.  All 
subjects  had  extensive  laparoscopic  experience 
(>30  cases)  and  40%  of  subjects  had  moderate 
experience  (>15  cases)  with  laparoscopic  US.  The 
RLUS  was  noted  by  the  majority  of  subjects  to  be 
associated  with  better  positioning  (8/10),  more 
comfortable  (6/10),  greater  confidence  in  lesion 
finding  (8/10),  less  fatigue  inducing  (9/10),  and 
an  overall  more  useful  tool  (9/10).  The  RLUS  was 
associated  with  less  fatigue,  increased  ergonomic 
comfort,  and  improved  lesion  identification 
among  the  subjects.  Overall,  9  out  of  10  subjects 
identified  the  robotic  US  as  the  more  useful 
imaging  tool  in  the  study.  A  sample  of  comments 
from  the  questionnaire  noted  that  subjects  “loved 
the  mapping  and  measurement  tools”  and  thought 
the  “design  was  excellent”;  however,  the  RLUS  was 
limited  by  “no  tactile  feedback”  and  “although 
scanning/ screening  was  more  rapid  with  the 
(handheld)  laparoscopic  US  device,  I  had  sense 
that  I  was/could  be  more  thorough  with  the 
robot.” 

DISCUSSION 

IOUS  plays  an  important  role  in  both  open  and 
laparoscopic  liver  surgery.  Studies  have  demon¬ 
strated  the  importance  of  IOUS  in  the  identifica¬ 
tion  of  occult  metastases  or  undetected  vascular 
invasion,  which  can  change  the  operative  plan  or 
identify  unresectable  disease  in  up  to  50%  of 
patients.21"23  The  recent  development  of 
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exploaration  Identification  Manipulation  punctures  per 

biopsy  attempt 
(x  10) 

Fig  3.  Comparison  of  the  da  Vinci  and  handheld  laparo¬ 
scopic  US  for  liver  surface  exploration,  intrahepatic  vas¬ 
cular  anatomy  identification,  tool  manipulation,  and 
number  of  punctures  needed  for  successful  biopsy 
(scaled,  X10,  s.d.). 

laparoscopic  and  robotic  liver  surgery  has  in¬ 
creased  the  importance  of  IOUS,  as  manual  palpa¬ 
tion  and  direct  visualization  are  impaired. 
Although  laparoscopic  IOUS  has  been  effective 
in  assessing  liver  for  operative  planning,  there 
are  some  constraints  to  its  use.24  The  limitations 
include  restricted  angulation,  torque  required  to 
maintain  surface  contact,  fewer  degrees  of  free¬ 
dom  for  orientation,  and  difficulty  in  optimizing 
probe  alignment  for  biopsy  or  ablation. 

Although  laparoscopic  liver  surgery  is  becoming 
an  increasingly  common  operative  approach  for  the 
management  of  various  benign  and  malignant 
tumors,  the  limitations  of  laparoscopic  US  are 
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restricting.  The  advantages  of  laparoscopic  liver 
surgery  parallel  those  seen  in  other  minimally 
invasive  operative  procedures,  including  decreased 
postoperative  pain,  lesser  duration  of  hospitaliza¬ 
tion,  and  shorter  incisions.25  Success  depends 
on  advanced  surgical  expertise  in  hepatobiliary  sur¬ 
gery  and  laparoscopic  surgery;  however,  the  inher¬ 
ent  ergonomic  restrictions  of  laparoscopy 
continue  to  limit  its  use  and  is  very  operator 
dependent. 

Robot-assisted  liver  surgery  allows  for  high- 
dexterity  control  and  3D  visualization,  and  retains 
the  advantages  of  minimally  invasive  access.  Al¬ 
though  robot-assisted  liver  surgery  may  potentially 
be  more  effective  than  the  laparoscopic  approach, 
it  is  similar  to  laparoscopic  liver  surgery  in  that  is 
relies  heavily  on  laparoscopic  US  for  intraoperative 
assessment  and  planning.  Certain  constraints  limit 
the  use  of  a  separate  laparoscopic  US  in  robotic 
operations,  including  lack  of  probe  mobility,  ma¬ 
neuverability,  and  image  incorporation  onto  exist¬ 
ing  displays.  With  the  absence  of  a  dedicated 
robotic  US,  currently  the  limitations  of  laparo¬ 
scopic  US  persist. 

We  have  developed  a  dedicated  robotic  US  and 
evaluated  its  utility  in  simulated  liver  surgery  tasks. 
With  the  loss  of  tactile  feedback  in  robotic  surgery, 
the  use  of  intraoperative  US  is  essential  for  opera¬ 
tive  planning.  The  robotic  US  device  was  developed 
as  an  integrated  instrument  with  those  previous 
limitations  in  mind.  Our  experiments  comparing 
the  robotic  US  with  conventional  laparoscopic 
US  found  the  robotic  US  to  be  more  ergonomic, 
associated  with  less  user  fatigue,  and  associated 
with  improved  liver  exploration  and  instrument 
manipulation.  Additionally,  we  have  developed  ad¬ 
ditional  features  for  the  robotic  US  system,  includ¬ 
ing  image  fusion  with  picture-in-picture,  robust 
measurement  tools,  and  dropped  virtual  markers, 
which  leverage  the  advantages  of  increased  inte¬ 
gration  and  kinematic  tracking.  Although  the  ro¬ 
botic  US  required  more  time  for  completion  of  one 
of  the  tasks,  further  studies  may  define  learning 
curve  improvements  and  whether  the  increase  in 
time  is  associated  with  improved  liver  US  explora¬ 
tion.  Future  studies  may  also  investigate  its  utility  in 
urologic,  gynecologic,  endocrine,  and  pancreatic 
robotic  procedures.  This  system  can  also  facilitate 
the  clinical  transition  of  more  advanced  IOUS 
imaging  tools  such  as  US  elastography  for  intra¬ 
operative  ablation  monitoring.26 

The  authors  acknowledge  Dr.  Emad  Boctor  for  his 
assistance  during  the  development  of  this  work. 
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Ultrasound  Elastography  Mosaicing 


Abstract.  Panoramic  ultrasound  imaging  is  emerging  as  a  prevalent 
technique  in  clinical  practice  with  a  high  clinical  value.  In  the  literature 
of  ultrasound  mosaicing,  registering  the  underlying  displacement  field 
for  elastography  has  not  yet  been  addressed.  The  clinical  advantages  of 
ultrasound  mosaics  can  be  improved  with  the  additional  corresponding 
strain  information.  In  this  paper,  we  propose  a  technique  for  generat¬ 
ing  a  reliable,  wide  field-of-view  displacement  field,  robust  to  sources  of 
decorrelation.  Elastography  mosaics  are  generated  from  two  pairs  of  ul¬ 
trasound  images,  and  then  from  multiple  image  pairs.  Tissue  mimicking 
phantom  data  is  used  for  the  validation  of  the  multi-image  elastography 
mosaic.  Finally,  the  method  is  extended  to  3D  ultrasound  elastography 
mosaicing  using  multiple  3D  volume  pairs. 


1  Introduction 

Compared  to  other  imaging  modalities  like  CT  and  MR,  ultrasound  suffers  from 
a  limited  field  of  view.  Monitoring  a  structure  can  be  particularly  challeng¬ 
ing  when  it  is  too  large  to  be  visualized  in  a  single  image  or  3D  sweep.  Size 
and  distance  measurements  are  unreliable  in  large  organs.  Panoramic  imaging  is 
emerging  as  a  prevalent  technique  used  in  widening  the  field  of  view  (FOV)  of 
medical  ultrasound  images.  Also  referred  to  as  stitching  or  panorama,  the  ultra¬ 
sound  mosaic  aims  to  achieve  several  clinical  advantages  which  come  along  with 
extended  FOV:  1)  improving  the  understanding  of  spatial  relationships  among 
structures  when  the  size  of  a  single  image/volume  is  not  large  enough  to  cover 
the  entire  area,  2)  allowing  for  measurements  of  size  and  distance  in  large  organs 
and  lesions,  3)  allowing  multi-modal  registration  and  fusion  with  pre-operative 
data  for  guidance  in  minimally-invasive  interventions. 

Ultrasound  elastography  is  a  technique  first  proposed  two  decades  ago  by  [1] 
for  the  clinical  imaging  of  tissue  stiffness.  Numerous  clinical  applications  have 
been  investigated,  among  them  cancer  imaging  [1],  ablation  monitoring  [2],  and 
the  detection  and  grading  of  deep  vein  thrombosis  [3] .  In  this  work  we  focus  on 
extended  FOV  displacement  estimation  for  quasi-static  ultrasound  elastography. 
The  tissue  is  imaged  while  it  is  slowly  deformed  using  an  external  mechanical 
force  and  the  images  are  used  for  the  estimation  of  tissue  motion  or  displacement 
[1].  Elastography  maps  the  mechanical  properties  of  tissue  which  can  add  valu¬ 
able  features  to  the  B-mode  panorama.  Many  clinical  applications  deal  with  large 
cancerous  lesions  which  expand  beyond  the  span  of  one  ultrasound  image  [4] .  An 
ultrasound  elastography  mosaic  can  improve  the  understanding  of  the  size  of  the 
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lesion  and  its  layout  among  surrounding  structures.  In  the  ablation  of  hepatic 
cancerous  tissue,  the  size  of  the  HIFU-induced  ablative  lesions  often  exceeds  4 
cm  in  diameter,  which  is  the  width  of  a  typical  ultrasound  transducer.  Ther¬ 
mal  lesions  are  not  visible  in  conventional  B-mode  ultrasound  but  a  panoramic 
ultrasound  elastogram  can  help  visualize  the  entire  extent  of  the  ablation  [2], 
monitoring  and  insuring  all  cancerous  tissue  is  ablated.  In  the  assessment  of  ve¬ 
nous  thrombi,  a  combination  of  ultrasound  B-mode  and  Doppler  imaging  help 
detect  the  presence  of  the  blood  clot,  but  it  is  elastography  which  can  provide 
its  age  and  clinical  grading  [3].  An  in-plane  ultrasound  elastography  mosaic  can 
provide  mapping  of  the  thrombi  all  along  the  femoral  vein.  It  could  take  up  to 
eight  mosaiced  volumes  to  depict  an  entire  organ  like  the  liver  or  kidney  [5] ,  but 
having  the  corresponding  elastography  mosaic  would  allow  for  registration  with 
pre-operative  imaging  data  (CT  or  MRI)  which  would  help  with  intra-operative 
navigation.  Although  the  basics  of  medical  elastography  have  long  been  defined, 
new  clinical  applications  are  constantly  emerging  are  we  are  seeing  an  increasing 
commercial  and  clinical  interest. 

Multiple  approaches  have  been  published  in  the  literature  on  2D  and  3D  wide 
FOV  ultrasound  mosaics  [6,5,7,  8],  but  very  little  of  it  concerns  registering  the 
underlying  strain  field.  As  various  sources  of  decorrelation  are  usually  affecting 
the  computation  of  strain  images,  this  problem  becomes  even  more  important 
when  attempting  to  generate  a  unified,  wide  displacement  field.  Further  more, 
most  elastography  algorithms  result  in  qualitative  strain;  2  image  pairs  with 
even  very  slightly  different  degrees  of  compression  will  produce  2  strain  images 
in  which  different  structures  could  be  visible.  Another  problem  rises  in  the  ambi¬ 
guity  of  the  interpretation  of  strain  images:  low  strain  can  be  indicative  of  high 
stiffness  but  this  interpretation  may  not  be  the  right  one  if  the  stress  field  is  not 
uniform  throughout  the  tissue  [1,9].  To  address  these  issues  and  to  improve  the 
quality  of  strain  images,  several  metrics  of  stability,  consistency /persistency  and 
reliability  have  been  developed  [10-12].  In  these  paper  we  propose  using  similar 
techniques  to  select  a  stable  pair  of  RF  lines  which  will  become  the  seed  for 
generating  a  reliable,  wide  FOV  displacement  field.  Displacement  on  the  seed 
line  is  calculated  using  dynamic  programming  and  later  propagated  in  both  lat¬ 
eral  directions  of  the  mosaic  using  an  analytic  minimization  approach  [13].  Each 
new  image  pair  adds  to  the  unified  displacement  field.  Here  we  present  tissue 
mimicking  phantom  data  for  2D  validation.  Finally,  the  method  is  extended  to 
3D  ultrasound  elastography  mosaicing  using  multiple  3D  volume  pairs. 

2  Methodology 

Consider  a  sequence  of  radio  frequency  (RF)  data  (/mu,  Im^)  collected  at 
position  to,  before  and  after  the  compression  of  tissue  using  a  2D  ultrasound 
transducer  (Fig.  lb).  Each  sequence  contains  n  RF  lines  of  length  m.  A  second 
sequence  (Im 21,  Im 22)  is  collected  (Fig.  lc)  after  the  transducer  has  been  moved 
in  the  lateral  direction  of  the  probe  to  position  ti,  with  the  help  of  a  moving  stage 
(Fig.  la).  A  vertical  stage  was  used  to  achieve  an  almost  identical  compression 


Ultrasound  Elastography  Mosaicing 


3 


rate  between  the  two  sequences.  The  translation  between  the  2  image  sequences 
is: 

Tiines  =  Trnrn  *  Tlj W  (1) 

where  w  is  the  width  of  the  ultrasound  transducer  in  millimeters  (mm),  n  is  the 
number  of  RF  lines,  Tmm  is  the  ground  truth  translation  in  mm  as  read  on  the 
stage  and  Tunes  is  the  corresponding  translation  as  number  of  RF  lines.  The 
overlap  area  consists  of  (n  -  Tunes)  RF  lines  (Fig.  Id). 


Fig.  1.  Experimental  setup  (a).  Bmode  ultrasound  data  before  and  after  compression 
for  (b)  position  to,  and  (c)  position  t\.  Translation  Tunes  and  the  length  of  the  overlap 
area  are  shown  in  (d). 


Pairwise  Mosaicing  In  Analytic  Minimization  (AM)  elastography  [13],  2D 
integer  displacements  are  first  obtained  using  dynamic  programming  (DP)  on 
a  single  pair  of  RF  lines  and  are  later  propagated  to  produce  2D  subsample 
displacements  for  the  entire  image.  For  pairwise  mosaicing  we  aim  to  find  a 
robust  and  stable  seed  RF  line  pair  in  the  overlap  area,  which  also  gives  consistent 
DP  integer  displacement  results  in  the  two  sequences.  The  displacement  estimate 
on  the  seed  line  will  serve  as  initial  guess  in  the  AM  propagation.  The  pairwise 
strain  mosaicing  is  implemented  as  follows: 

1.  Robustness  and  stability  measures  are  used  to  identify  in  the  overlap  area 
a  suitable  seed  RF  line  for  the  mosaicing  algorithm,  similarly  to  [12].  The 
process  filters  those  candidates  which  exhibit  a  high  degree  of  decorrelation 
(Figure  2): 

—  k  =  5  (ii,  ^2,  25)  random  corresponding  positions  (RF  line  pairs)  are 

selected  from  the  overlap  area  (Fig.  3-1). 

—  Compute  integer  axial  displacement  for  each  RF  line  pair  using  the  dy¬ 
namic  programming  approach  [13]:  (a\,  a3j, a\)  and  (a\ ,  a|, ...,  a§)  (Fig. 
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3-2).  Note:  in  this  step,  both  axial  and  later  integer  displacements  are 
computed,  but  from  here  on  only  axial  displacement  values  will  be  used. 

—  In  a  continuous  piece  of  tissue,  the  axial  displacement  profile  resulting 
from  a  stress  field  induced  by  applied  compression  has  a  monotonous 
ramp  [14].  A  pair  of  decorrelated  RF  lines  would  affect  the  DP  displace¬ 
ment  estimation  algorithm.  Instead  of  a  globally  optimal  solution,  the 
estimate  could  exhibit  regions  of  locally  optimal  solutions.  For  small 
deformations,  these  locally  optimal  solutions  would  correspond  to  per¬ 
turbations  in  the  monotonously  decreasing  slope  of  their  displacement 
profile,  which  in  turn  could  result  in  artifacts  in  the  final  strain  image. 
If  three  or  more  positions  along  the  displacement  profile  would  exhibit 
a  continuous  change  in  the  slope  of  displacement,  this  could  point  to  a 
region  of  poorly  correlated  data. 

—  For  the  remaining  positions  which  satisfy  the  monotonous  slope  criterion, 

dk  =  abs(a\  —  a|);  avg k  =  mean(dk)',  stdk  =  stdev(dk)  (2) 

We  select  the  seed  RF  line  s  as  the  position  with  the  minimum  stdk  and 
avgk  <  1. 

—  At  the  end  of  the  subinteger  displacement  estimation  process  on  the  a 
pair  of  RF  lines,  A{  =  ai  +  Aai,  Li  =  U  +  Ali  are  the  axial  and  lateral 
displacements  at  each  sample  i.  The  magnitude  of  the  normalized  cross¬ 
correlation  could  be  used  to  assess  the  degree  of  matching  between  the 
pre-  and  post-compression  data. 


C(h(s)J2(s)) 


'Z^1{h{i,s)-T1){I2(.i+Ai,s+Li)-T2) 


(3) 


where  I\  and  I2  are  the  means  of  RF  values  along  the  seed  line  s.  The 
value  of  C  could  be  used  as  an  additional  check  to  verify  if  the  displace¬ 
ment  estimate  for  the  selected  seed  line  is  a  good  fit.  If  C  value  would 
fall  below  a  certain  threshold  (to  be  established  by  the  user),  the  seed 
line  would  be  deemed  not  suitable  and  the  next  stable  seed  line  would 
be  selected  (Fig.  3-3). 

2.  Subinteger  displacement  values  are  calculated  for  seed  line  s  (Fig.  3-4). 

3.  The  subinteger  displacement  values  are  propagated  using  AM: 

—  in  one  sequence  towards  the  first  RF  line,  and 

—  in  the  second  sequence  towards  the  last  RF  line. 

The  unified  displacement  field  will  have  n  +  Tunes  RF  lines  and  the  stitch 
will  be  around  RF  line  number  s  +  Tunes  (Fig.  3-5). 

4.  Generate  the  ultrasound  elastography  mosaic  from  the  unified  displacement 
field  (Fig.  3-6). 


Multi-image  Mosaicing  Given  an  existing  nt  wide  displacement  map,  com¬ 
puted  for  image  pairs  1  through  £,  we  expand  our  displacement  map  to  include 
information  from  image  pair  t  +  1  as  follows: 
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Fig.  2.  Robust  seed  selection  method. 


1. 

2. 

3. 


4. 


Compute  the  translation  T^es  between  position  t  and  position  t  + 1  and  the 
overlap  area. 

Select  the  seed  RF  line  st+i  in  the  new  overlap  area. 

Use  the  subinteger  values  form  the  wide  FOV  displacement  map  at  RF  line 
number  st+i  +  Tunes  as  seed  values  to  propagate  using  AM: 

—  in  the  new  sequence  towards  the  first  RF  line. 

Stitch  the  newly  computed  (st+i  +  Tunes)  lines  at  the  beginning  of  the 
previous  displacement  map.  The  new  mosaiced  displacement  field  will  have 
nt  +  Tf^ls  RF-lines. 

Generate  the  new  ultrasound  elastography  mosaic  from  the  updated  dis¬ 
placement  field. 


Multi-volume  Mosaicing  In  the  three-dimenssional  case,  consider  a  pair  of 
RF  volumes  (Vh,  V12)  collected  at  position  to,  before  and  after  the  compression 
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Select  k=5  EF  line  pairs  Compute  integer  displacements  seed  RF-line  s 


Subinteger  displacement  Displacement  propagation 


Stitching 


Elastography  mosaic 


4) 


5) 


6) 


Fig.  3.  Pairwise  Mosaicing  workflow:  k  =  5  random  RF  positions  are  selected  (1),  in¬ 
teger  displacement  is  calculated  for  each  position  (2),  the  seed  RF-line  s  is  selected 
(3),  subinteger  displacement  values  are  calculated  for  s  (4),  propagated  in  the  2  se¬ 
quences  and  stitched  (5).  The  ultrasound  elastography  mosaic  is  generated  from  the 
displacement  field  (6). 


of  tissue  using  a  3D  ultrasound  wobbler  transducer.  Each  volume  contains  k 
frames  and  each  frame  has  n  RF-lines  of  length  m.  One  or  more  subsequent 
pairs  of  volumes  are  collected  after  the  wobbler  transducer  is  moved  in  the  lateral 
direction  of  the  probe  with  the  help  of  a  moving  stage.  Once  again,  a  vertical 
stage  was  used  to  achieve  an  almost  identical  compression  rate  between  the  two 
sequences.  Using  the  lateral  and  axial  stages  ensures  the  first  frame  of  the  first 
volume  pair  is  alligned  with  the  first  frame  of  each  subsequent  pair  volume,  and 
all  the  other  frames  are  simillarly  aligned,  with  no  out-of-plane  motion  between 
each  position.  The  same  steps  of  multi-image  mosaicing  can  be  applied  on  a 
frame  by  frame  basis,  resulting  in  a  unified  displacement  field  for  each  elevational 
position  of  the  wobbler  motor. 

Note:  here  we  present  a  very  controlled  way  of  acquiring  3D  ultrasound  data 
for  the  purpose  of  building  3D  strain  mosaics.  In  a  routine  3D  ultrasound  exam, 
using  stages  to  control  the  imaging  of  a  certain  human  tissue  is  absolutely  not 
feasibile.  This  does  not,  however  invalidate  our  framework.  As  electromagnetic 
trackers  become  integrated  into  ultrasound  transducers,  it  is  feasibile  to  select  ul¬ 
trasound  volumes  consisting  only  of  aligned  frames  with  no  out-of-plane  rotation. 
The  work  presented  here  makes  a  case  for  in-plane  mosaicing  where  translation  is 
the  only  component  of  the  transformation.  One  can  easily  envision  an  ultrasound 
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system  equipped  with  a  tracked  transducer  like  in  the  Foroughi  set-up  [15].  Here 
we  used  a  controlled  setup  for  proof  of  concept,  but  using  the  Foroughi  method 
[15]  one  could  achieve  a  3D  elastography  mosaic  as  follows: 

1.  Perform  freehand  palpation  with  tracked  3D  transducer, 

2.  Use  the  tracking  information  [15]  to  select  suitable  aligned  frames  separated 
only  by  in-plane  translation, 

3.  Apply  2D  multi-image  mosaic  method  on  a  frame-by-frame  basis  to  estimate 
a  unified  displacement  field,  and 

4.  Generate  the  3D  elastography  mosaic  from  the  unified  displacement  field. 

3  Results  and  Validation 

For  experimental  validation  we  palpated  a  CIRS  (Norfolk,  VA)  elastography 
phantom  049a  (Fig.  la)  using  a  high-frequency  ultrasound  transducer  (L14- 
5W/60)  at  center  frequency  of  10  MHz.  Ultrasound  RF  data  was  acquired  from 
an  Ultrasonix  system  (Vancouver,  BC)  at  40MHz  sampling  rate.  The  049a  phan¬ 
tom  consists  of  a  series  of  stepped  cylinders  of  varying  diameters.  The  transducer 
was  placed  on  top  of  one  of  the  cylinders,  parallel  with  its  direction.  The  phan¬ 
tom  was  placed  on  a  stage  which  controlled  the  compression  in  the  axial  direction 
and  the  translation  in  the  lateral  direction  (Fig.  la).  RF  data  sequences  were 
acquired  from  four  axial  compression  levels  (co  =  0,  c\  —  2.54mm,  C2  =  5.08mm, 
C3  =  back  to  0),  for  each  of  four  lateral  translation  positions  of  the  transducer 
(to  =  0,  t\  =  5.08mm,  t2  =  10.16mm,  ts  =  20.32mm),  for  a  total  of  sixteen 
data  sets.  The  width  of  the  ultrasound  transducer  L14-5W  is  59mm,  and  as  a 
consequence  any  2  sets  separated  by  a  lateral  translation  had  some  amount  of 
overlap. 


Fig.  4.  Ultrasound  elastography  at  position  to  (a),  ti  (b),  (c),  1 3  (d)  and  elastography 

mosaic  of  positions  to  and  ti  (e),  to,  ti  and  1 2  (f),  and  to,  ti,  t2  and  1 3  (g). 
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Ultrasound  elastography  images  were  obtained  for  each  translation  position 
to  -  £3  ((Fig.  4a-d).  A  pairwise  elastography  mosaic  was  produced  from  positions 
to  and  ti  ((Fig.  4e),  and  then  multi-image  elastography  mosaics  from  positions 
to,  h  and  t2  ((Fig.  4f),  and  t0,  h,  t2  and  t3  ((Fig.  4g). 

Mosaicing  multiple  images  in  a  pairwise  fashion  can  cause  errors  to  accu¬ 
mulate  with  each  new  added  image.  To  calculate  this  error  we  computed  the 
wide  FOV  displacement  map  from  positions  to  and  £3,  and  compared  it  with 
the  multi-image  displacement  map  from  positions  to,  £1,  £2  and  £3  added  con¬ 
secutively.  The  random  search  algorithm  was  run  5  times  and  the  results  were 
averaged.  For  the  resulting  displacement  maps,  we  computed  an  absolute  differ¬ 
ence  per  pixel,  as  well  as  mean  and  standard  deviation  absolute  difference  for 
the  entire  FOV. 


Fig.  5.  Panoramic  displacement  map  of  positions  to  and  t3  (a),  and  of  positions  to,  £1, 
t2  and  £3  (b),  the  absolute  difference  between  them  (c),  and  the  mean  and  standard  de¬ 
viation  of  this  difference,  per  line  (f).  The  unit  of  displacement  is  pixels.  Corresponding 
ultrasound  elastography  mosaics  of  positions  to  and  1 3  (d),  and  to,  £1,  t2  and  t3  (e). 


Figure  5  shows  the  results  of  the  validation  experiment.  The  two  panoramic 
displacement  maps  agree  almost  exactly;  the  absolute  difference  for  the  entire 
image  had  a  mean  of  0.0755  and  a  standard  deviation  of  0.2386  pixels.  The 
maximum  difference  was  2.0168  pixels  and  it  did  not  exceed  1  pixel  on  a  mean, 
per-line  basis.  For  a  metric  correspondent,  2.0168  pixels,  is  the  equivalent  of 
0.0406  mm  in  the  axial  direction.  Once  the  displacement  maps  are  converted  in 
elastogram  mosaics  (Fig.  5  e,f),  the  difference  becomes  indistinguishable  even  on 
close  inspection.  With  the  clinical  application  in  mind,  we  conclude  that  these 
small  differences  are  acceptable. 
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4  3D  mosaicing 

For  3D  validation,  RF  data  was  acquired  using  an  Ultrasonix  system  (Vancou¬ 
ver,  BC)  at  20MHz  sampling  rate.  The  same  CIRS  (Norfolk,  VA)  elastography 
phantom  049a  (Fig.  la)  was  palpated  with  a  38mm  width  linear  4D  volumetric 
transducer  (4DL 14-5/38).  As  with  the  2D  experiment,  the  phantom  was  placed 
on  a  stage  to  controll  the  compression  in  the  axial  direction  and  the  translation 
in  the  lateral  direction  (Fig.  1  a).  RF  data  sequences  were  acquired  from  four 
axial  compression  levels  (co  =  0,  c\  =  2.54mm,  c 2  =  5.08mm,  C3  =  back  to  0),  for 
each  of  five  lateral  translation  positions  of  the  transducer  (to  =  0,  t\  =  5.08mm, 
£2  =  10.16mm,  £3  =  20.32mm,  £4  =  22.86mm),  for  a  total  of  twenty  data  sets. 

Figure  6  shows  the  results  of  the  3D  ultrasound  elastography  mosaic.  Strain 
in  the  middle  frames  of  the  axial-lateral  plane  (Figure  6a),  lateral-elevational 
plane  (Figure  6b),  and  3  axial-elevational  planes  (Figure  6c-e)  is  shown.  The 
diameters  of  the  stiff  cyclinder  along  the  mosaicing  direction  were:  4.05  mm 
(Figure  6c),  2.53  mm  (Figure  6d)  and  1.58  mm  (Figure  6e). 


Fig.  6.  3D  ultrasound  elastography  mosaic:  axial- lateral  plane  (a),  lateral-elevational 
plane  (b),  and  3  axial-elevational  planes  through  3  different  diameters  of  the  stepped 
cylinder  (c,d,e) 


5  Conclusion 

We  have  presented  an  algorithm  for  generating  reliable  multi-image  ultrasound 
elastography  mosaics,  robust  to  regions  of  decorrelation.  Panoramic  B-mode  ul¬ 
trasound  has  been  reported  with  success  in  the  literature  and,  with  the  addi¬ 
tion  of  corresponding  elastograms,  an  ultrasound  system  with  these  capabilities 
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has  great  potential  in  clinical  diagnostic  and  monitoring.  Elastography  has  the 
advantage  that  is  requires  no  additional  hardware  to  be  implemented,  and  a 
majority  of  commercial  ultrasound  systems  now  present  an  elastography  inter¬ 
face.  Furthermore,  the  challenge  of  mosaicing  while  freehand  scanning  can  be 
met  as  efforts  are  under  way  to  incorporate  electromagnetical  (EM)  tracking  in 
ultrasound  transducers. 
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ABSTRACT 

Ultrasound  elastography  is  an  imaging  technology  which  can  detect  differences  in  tissue  stiffness  based  on  tissue 
deformation.  For  successful  clinical  use  in  cancer  diagnosis  and  monitoring  the  method  should  be  robust  to  sources 
of  decorrelation  between  ultrasound  images.  A  regularized  Dynamic  Programming  (DP)  approach  was  used  for 
displacement  estimation  in  compressed  tissue.  In  the  Analytic  Minimization  (AM)  extension  of  DP,  integer 
displacements  are  calculated  just  for  one  RF-line,  and  later  propagated  laterally  throughout  the  entire  image. 
This  makes  the  seed  RF-line  very  important;  faulty  seed  lines  could  propagate  erroneous  displacement  values 
throughout  the  image  resulting  in  the  appearance  of  false  ”  lesions” .  In  this  paper  we  analyze  the  robustness  of 
this  method  in  free-hand  palpation  of  laboratory  tissue  phantoms.  We  are  proposing  an  update  to  the  algorithm 
which  includes  a  random  search  for  the  most  robust  seed  RF-line.  Axial  integer  displacements  are  obtained 
on  each  random  seed  line  individually  with  DP  optimization.  For  each  random  axial  RF-line,  multiple  random 
values  for  decorrelation  compensation  are  used  in  the  displacement  estimation.  The  displacement  values  are  then 
compared  and  several  metrics  of  stability  and  consistency  are  considered.  A  ranking  is  established  and  the  line 
deemed  most  robust  will  become  the  seed  line  for  displacement  propagation,  while  also  selecting  the  most  stable 
value  for  decorrelation  compensation.  The  random  search  can  be  achieved  at  no  additional  computational  cost 
in  a  parallel  implementation.  The  results  indicate  significant  improvement  in  the  robustness  of  the  DP  approach, 
while  maintaining  real-time  computation  of  strain  images. 

1.  BACKGROUND  AND  RATIONALE 

Ultrasound  Elastography  is  an  imaging  technology  which  can  detect  differences  in  tissue  stiffness  based  on  tissue 
deformation.1  Our  work  in  this  paper  focuses  on  real-time  quasi-static  elastography.  The  tissue  is  compressed 
and  relaxed  in  a  continuous  free-hand  motion  and  ultrasound  images  are  simultaneously  acquired.  This  method 
is  easy  to  use  and  also  cheap  as  it  requires  no  extra  hardware.  This  makes  it  particularly  appealing  for  medical 
imaging  applications;  diagnosis  or  monitoring  could  be  done  at  the  patient’s  bed  side.  There  are  two  hurdles 
which  once  resolved  would  ensure  success  for  clinical  use:  real-time  computation  of  strain  images  and  dealing  with 
the  potential  for  global  and  local  decorrelation  between  pre-  and  post-compression  ultrasound  signals.  Various 
sources  of  decorrelation  are  affecting  the  computation  of  strain  images  in  in-vivo  data,  such  as  incoherent  fluid 
(blood)  motion,  out-of-plane  motion  of  structures  within  one  image  due  to  transducer  or  respiratory  motion, 
subsample  speckle  motion,  and  a  high  degree  of  compression. 

Rivaz  and  Hall  initially  proposed  optimizing  a  recursive  regularized  cost  function  using  Dynamic  Programming 
(DP).2,3  The  method  resulted  in  integer  values  for  axial  displacement  and  subsample  displacement  could  also  be 
achieved  but  at  a  high  computational  cost.  Rivaz  et  al.  refined  the  method  further  using  a  ID  and  2D  Analytic 
Minimization  (AM)  of  the  cost  function.4  It  takes  the  integer  displacement  of  a  single  axial  radio  frequency 
(RF)-line  from  DP  and  produces  the  subsample  axial  and  lateral  displacement  fields  for  the  entire  image. 

In  this  work  we  analyze  the  robustness  of  the  AM2D  method.  Since  this  method  is  based  on  computing 
integer  displacements  using  DP  on  only  one  RF-line,  it  is  crucial  that  we  choose  a  starting  line  with  little  or  no 
decorrelation  between  the  two  images.  Here  we  present  a  method  for  identifying  robust,  stable  lines  and  reducing 
the  potential  to  generate  artifact  lesions. 


2.  2D  AM  SUBPIXEL  DISPLACEMENT  ESTIMATION 


Here  we  briefly  review  the  2D  AM  method  in  which  2D  integer  displacements  are  first  obtained  using  DP  on  a 
single  RF-line  and  are  then  used  to  produce  2D  subsample  displacements  for  the  entire  image. 

1.  Calculate  integer  axial  cq  and  lateral  li  displacements  of  one  seed  RF-line  using  DP  (2).  Calculate  an  initial 
subsample  estimate  using  linear  interpolation  of  the  integer  displacements. 


Let  ii  and  I2  be  two  ultrasound  images  acquired  before  and  after  deformation.  Let  m  be  the  number  of 
RF-lines.  Each  signal  is  sampled  at  i  =  1,  2  •  •  •  m.  A  regularized  cost  function  is  generated  combining  the 
prior  of  displacement  continuity  (regularization  term)  and  an  amplitude  similarity  term.  The  displacement 
continuity  term  for  line  j  can  be  written  as: 

Rj  —  !•>  li—  l)  ^i— l)  T  Oil  (li  — l)  (1) 

where  aa  and  cq  are  axial  and  lateral  regularization  weights  respectively. 


The  regularized  cost  function  at  the  ith  sample  of  the  j th  A-line  becomes: 


Cj(a,i,li,i)  =  [h (i,j)  -  I2(i  +  a,i,  j  +  k)]2  +mmda4l  j 


Cj(da:di:i  1)  ~\~Cj  —  i  (da  :di  ,z) 


T  wRj  (cLi ,  li  ,  da ,  di)} 


(2) 

where  re  is  a  regularization  weight  for  smoothness;  da  and  di  are  temporary  axial  and  lateral  displacements 
which  are  varied  in  order  to  minimize  eqn.  (2). 


2.  Calculate  subsample  axial  and  lateral  displacements  for  the  seed  RF-line  using  2D  AM  (below).  They  will 
be  added  to  the  initial  integer  estimates. 


3.  Propagate  the  solution  of  the  seed  RF-line  to  the  left  and  right,  using  the  displacement  of  the  previous 
line  as  initial  estimate. 

The  aim  is  to  calculate  A  eq  and  A  li  such  that  the  duple  (cq  +  Aa^Z^  +  A  li)  gives  the  axial  and  lateral 
displacements  at  the  sample  i.  The  regularized  cost  function  becomes: 

Cj(Aai, ...,  A am,  AZi, ...,  A Zm)  =  Ylln  1{[^i(bi)  —  hi'l  +  ca  +  Aa^,  j  +  k  +  A Z^)]2  +  /^\ 

-\-a  (a>i  +  Aa^  —  a^_  i  —  Atq_i)  +  (3a  (li  +  A  li  —  Z^_i  —  AZ^_i)  +  ^  (li  +  A  li  —  Uj- 1)  } 

, where  the  index  j  was  dropped  for  the  jth  RF-line  and  kj-i  is  the  lateral  displacement  of  the  previous 
RF-line  (except  for  the  seed  line  where  hj-i  =  h-  OL,(da  and  / 3t  are  regularization  terms  which  ensure 
continuity  in  displacements  with  respect  to  the  top  (axial  a),  and  the  top  and  left /right  (lateral  (3a  and 
/3t).  If  the  displacement  of  the  previous  line  is  not  accurate,  it  will  affect  the  displacement  of  the  next  line 
through  the  last  term  in  the  right-hand  side  of  (3). 


3.  EXPERIMENTAL  DESIGN  AND  RESULTS 

For  experimental  evaluation  we  palpated  a  breast  elastography  phantom  with  a  10mm  lesion  and  three  times 
stiffer  than  the  background.  RF  data  was  acquired  with  a  7.27MHz  linear  array  at  a  sampling  rate  of  40MHz. 
In  order  to  evaluate  the  magnitude  of  the  problem,  we  first  set  out  to  evaluate  the  percentage  of  seed  RF-lines 
with  faulty  DP  displacement  estimations.  When  propagated  laterally  to  the  neighboring  lines,  these  faulty 
displacement  estimations  would  result  in  artifact  lesions,  clearly  visible  on  the  final  elastogram  (Fig.  1).  Many 
of  the  artifacts  created  by  the  erroneous  displacement  estimation  were  very  small  (Fig.  1  b)  and  localized  at  the 


(d)  (e)  (f) 


Figure  1.  Examples  of  artifact  lesions  in  strain  images  using  DP  AM2D  method.  Data  collected  from  breast  phantom 
freehand  palpation. 


top  of  the  image  (Fig.  1  b,  c).  Some  artifacts  were  however  large,  sometimes  propagating  through  a  big  part  of 
the  image  (Fig.  1  d,  e)  and,  in  very  rare  cases,  obscuring  the  real  lesion  (Fig.  1  f). 

DP  displacement  computation  uses  a  smoothness  regularization  parameter  w  (2),  which  should  prevent  regions 
with  high  local  decorrelation  from  introducing  errors  in  displacement  estimation,  but  if  chosen  too  large  would 
result  in  oversmoothing.  Strain  images  were  obtained  with  the  AM2D  method  using  each  RF-line  as  a  seed,  each 
for  11  (eleven)  values  for  w:  0.1,  0.15,  0.2,  0.25,  0.3,  0.35,  0.4,  0.45,  0.5,  0.55  and  0.6.  Each  resulting  elastogram 
was  visually  inspected  for  the  presence  of  artifacts,  knowing  the  shape  and  size  of  the  expected  lesion.  The 
percentages  of  faulty  and  good  seed  lines  are  summarized  in  Table  1.  Some  lines  produced  very  faint,  very 
small  artifacts  on  the  order  of  a  couple  of  pixels  which  were  not  clearly  visible  at  first  inspection  and  they  were 
categorized  as  indeterminate. 

Table  1.  Percentage  distribution  of  AM2D  behavior  for  seed  RF-lines  given  multiple  values  for  w  (smoothness  regularization 
parameter) 


w 

Good  Lines  (%) 

Faulty  Lines  (%) 

Indeterminate  (%) 

0.10 

44.2 

55.8 

- 

0.15 

57.8 

42.2 

- 

0.20 

61.3 

33.6 

5.1 

0.25 

68.1 

30.0 

1.9 

0.30 

65.0 

30.5 

4.5 

0.35 

61.7 

32.0 

5.3 

0.40 

53.3 

41.8 

4.9 

0.45 

53.1 

46.9 

- 

0.50 

43.4 

56.6 

- 

0.55 

32.1 

67.9 

- 

0.60 

24.3 

75.8 

- 

The  results  were  very  revealing  both  about  the  importance  of  the  value  of  w,  but  also  of  the  magnitude  of 
the  decorrelated  areas.  We  noted  that  faulty  lines  did  not  appear  to  be  predominant  in  certain  locations,  like 
for  example  towards  the  lateral  margins  of  the  imaged  area,  but  they  were  actually  widely  dispersed  throughout 
the  image.  In  light  of  the  results,  for  this  specific  data  set  we  continued  our  investigation  only  for  w  in  the  range 
(0.25,0.35),  to  ensure  a  high  probability  of  finding  a  good  starting  RF-line. 


Integer  Displacement  Profile  for  Seed  RF-lines  249  to  255 


Figure  2.  Integer  DP  displacement  estimation  for  seed  RF-lines  249  to  255.  Note  the  areas  (for  lines  250  and  252 
respectively)  where  the  change  in  slope  produces  the  artifact  lesions 

3.1  Deformation  Slope 

In  a  continuous  piece  of  tissue,  the  deformation  field  resulting  from  a  stress  field  induced  by  applied  compression 
has  a  monotonous  ramp  profile.5  Our  hypothesis  was  that  faulty  lines  would  exhibit  a  perturbation  in  the 
monotonously  decreasing  slope  of  their  displacement  profile.  For  example,  when  plotting  DP  displacement 
values  for  seed  RF-lines  249  to  255  (Fig.  2),  faulty  lines  250  and  252  respectively  exhibited  a  change  in  slope 
towards  the  top  portion  of  the  imaging  area,  which  in  the  end  resulted  in  artifact  lesions  (Fig.  1  c  -  250  and  b  - 
252). 

A  chang e-in- slope  parameter  was  computed  for  the  DP  displacement  profile  for  each  seed  RF-line,  for  w= 
0.25,  0.30, 0.35.  486  lines  were  evaluated  for  each  of  the  three  values  for  w,  for  a  total  of  1458  computations.  The 
chang  e-in- slope  test  was  considered  positive  when  the  3  (three)  or  more  positions  exhibited  a  continuous  change 
in  the  slope  of  displacement.  Note  that  the  3  (three)  positions  did  not  need  to  be  consecutive,  as  long  as  the 
ramp  stayed  flat  or  continued  the  change  direction(Fig.  3  a,  b). 

3.2  Displacement  Stability 

Another  measure  of  robustness  is  stability.  A  robust  seed  RF-line  needs  to  exhibit  the  same  or  similar  DP 
displacement  estimation  across  different  w  values.  We  computed  the  percentage  of  positions  (pixels)  for  which 
displacement  values  differed  more  than  3  (three)  pixels  between  different  values  of  w.  For  all  486  lines,  3  (three) 
pairs  of  displacement  values  were  compared:  1)  w  =  0.25  vs.  w  =  0.30,  2)  w  =  0.30  vs.  w  =  0.35,  3)  w  =  0.25  vs. 
w  =  0.35.  The  score  received  by  each  line  was  averaged  over  the  three  comparisons  for  a  final  score,  potentially 
ranging  from  0  to  1  (100%  of  pixels  exhibited  more  than  3  (three)  pixels  differences).  We  wanted  to  test  the 
statistical  significance  of  the  difference  between  the  faulty  lines  population  and  the  robust  lines  population. 


Figure  3.  Chang  e-in- slope  detection  algorithm.  The  3  (three) 
consecutive  (a)  or  not  (b) 


positions  where  the  change  in  slope  is  exhibited  could  be 


4.  RESULTS  AND  DISCUSSION 

4.1  Deformation  Slope 

For  a  robust  AM2D  algorithm  one  single  successful  seed  RF-line  is  sufficient.  The  chang e-in- slope  parameter 
was  designed  to  select  good  seed  lines  with  monotonously  increasing/decreasing  displacement  profile.  Over  all 
1458  computations  of  the  parameter  (486  lines  *3  w  values),  the  sensivity  for  selecting  a  good  line  was  91.7%, 
while  the  specificity  was  48.6%  (  Table  2).  The  measure  performed  up  to  93.1%  sensitivity  when  tested  across 
just  one  w  value.  We  concluded  the  measure  was  promising  but  not  sufficient  on  its  own. 


Table  2.  Chang  e-in- Slope  (TP  =  true  positive,  FP  =  false  positive,  TN  =  true  negative,  FN  »  false  negative) 

Prediction  outcome 


actual 

value 


p 

n 

TP  =  835 

FN  =  238 

FP  =  76 

TN  =  252 

P 

N 

total 

P' 


N' 


4.2  Displacement  Stability 

486  scores  were  computed,  one  for  each  potential  seed  RF-line.  The  scores  ranged  from  0  to  0.2649.  The  average 
score  for  the  good  lines  was  0.0058  (stdev  =  0.0186)  and  the  average  score  for  the  faulty  lines  was  0.0444  (stdev 
=  0.0586).  A  one-side  Student  t-test  showed  a  p- value  of  3.73909e  —  17  (  Table  3). 

To  increase  the  significance  of  the  prediction  value,  we  also  computed  a  combined  score  for  the  two  detection 
algorithms.  The  averaged  Chang  e-in- Slope  score  over  the  3  (three)  values  for  w  was  either  0,  0.33,  0.66,  or 
1,  depending  on  how  many  of  the  three  instances  were  deemed  positive  by  the  test.  This  score  was  added  to 
the  Displacement  Stability  score  for  a  combined  overall  score.  This  overall  score  ranged  from  0  to  1.1588.  The 
average  score  for  the  good  lines  was  0.0795  (stdev  =  0.2422)  and  the  average  score  for  the  faulty  lines  was  0.4939 
(stdev  =  0.4444),  for  a  p-value  of  1.75837e  —  27  (  Table  3). 


Table  3.  Robustness  score:  Displacement  Stability  vs.  Combined  score  for  Displacement  Stability  +  Change- in-Slope 


Displ.  Stability 

Displ.  Stability  +  Change-in-Slope 

Good  Lines 

Faulty  Lines 

Good  Lines 

Faulty  Lines 

Min 

0 

0 

0 

0 

Max 

0.1002 

0.2649 

1 

1.1588 

Avg 

0.0058 

0.0444 

0.0795 

0.4939 

StDev 

0.0186 

0.0586 

0.2422 

0.4444 

p- value 

3.73909E-17 

1.75837E-27 

Following  the  results  of  this  analysis,  a  random  search  algorithm  was  implemented  for  the  selection  of  a 
robust,  stable  seed  RF-line  as  follows: 

1.  DP  integer  displacement  is  calculated  for  5  random  RF-lines,  each  at  5  random  w  values 

2.  A  combined  Chang  e-in- Slope  average  plus  Displacement  Stability  average  score  is  computed 

3.  The  most  robust,  stable  line  is  chosen  as  the  line  with  the  smallest  combined  score,  which  also  does  not 
have  any  positive  Chang  e-in- Slope  score 

4.  The  chosen  seed  line’s  displacement  values  are  propagated  using  the  AM2D  method 

Given  the  current  parallel  computational  resources,  the  addition  of  this  selection  test  does  not  add  a  significant 
amount  of  time  to  the  overall  running  time.  DP  takes  the  same  time  as  before  but  it’s  now  computed  25  times, 
and  the  computation  of  each  score  takes  on  the  order  of  a  couple  of  milliseconds.  On  100  random  runs  of  our 
testing  algorithm,  we  only  encountered  1  (one)  situation  where  a  faulty  line  was  selected.  The  reason  for  the 
selection  was  that  all  5  of  the  random  line  tested  were  faulty.  This  prompted  us  to  introduce  the  additional 
condition  that  the  chosen  line  should  not  have  any  positive  Chang  e-in- Slope  score.  Following  this  modification, 
no  faulty  situation  has  been  encountered  so  far.  We  will  continue  to  test  our  algorithm  on  the  presented  data 
set,  as  well  as  on  new  ex-vivo  and  in-vivo  tissue  data. 

5.  CONCLUSION 

We  proposed  and  successfully  implemented  an  algorithm  for  the  selection  of  a  robust,  stable  RF-line  to  be  used  as 
seed  for  the  DP  displacement  estimation  and  later  propagated  using  the  AM2D  algorithm  for  elastography.  The 
benefit  of  this  algorithm  is  significant  as  it  has  the  potential  to  improve  the  robustness  of  ultrasound  elastography 
in  in-vivo  tissue  which  can  be  highly  decorrelated.  We  are  in  the  process  of  evaluating  this  hypothesis.  The 
selection  of  robust  seed  RF-lines  becomes  even  more  important  as  we  move  towards  real-time  3D  ultrasound 
elastography. 
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f' ABSTRACT:  Ultrasound  elastography  is  an  imaging  technology  which  can  detect  differences  in  tissue  stiffness  based  on  tissue  deformation.  For  successful  clinical  use  in  cancer  diagnosis  and  monitoring  the  method  should  be  robust  to  sources  of  decorrelation 
between  ultrasound  images.  A  regularized  Dynamic  Programming  (DP)  approach  was  used  for  displacement  estimation  in  compressed  tissue.  In  the  Analytic  Minimization  (AM)  extension  of  DP,  integer  displacements  are  calculated  just  for  one  RF-line,  and  later 
propagated  laterally  throughout  the  entire  image.  This  makes  the  seed  RF-line  very  important;  faulty  seed  lines  could  propagate  erroneous  displacement  values  throughout  the  image  resulting  in  the  appearance  of  false  "lesions".  In  this  paper  we  analyze  the 
robustness  of  this  method  in  free-hand  palpation  of  laboratory  tissue  phantoms.  An  update  to  the  algorithm  includes  a  random  search  for  the  most  robust  seed  RF-line.  Axial  integer  displacements  are  obtained  for  5  random  lines  individually  with  DP  optimization. 
For  each  random  axial  RF-line,  multiple  random  values  for  decorrelation  compensation  are  used  in  the  displacement  estimation.  The  displacement  values  are  then  compared  and  several  metrics  of  stability  and  consistency  are  considered.  A  ranking  is  established 
and  the  line  deemed  most  robust  will  become  the  seed  line  for  displacement  propagation,  while  also  selecting  the  most  stable  value  for  decorrelation  compensation.  The  random  search  can  be  achieved  at  no  additional  computational  cost  in  a  parallel 
implementation .  The  results  indicate  significant  improvement  in  the  robustness  of  the  DP  approach,  while  maintaining  real-time  computation  of  strain  images. 


INTRODUCTION: 

-  Take  one  image  before  compression, 

-  Compress  tissue  and  take  second  image, 

-  Compare  images  and  calculate  displacement 
(tissue  motion)  field, 

-  Calculate  strain  image  from  the  motion  field 

imu 

B-mode  Motion  field  Strain 

DYNAMIC  PROGRAMMING  (DP)  +  ANALYTIC  MINIMIZATION  (AM): 

1. For  images  /,  and  I2,  calculate  integer  axial  at  and  lateral  /,  displacements 
of  one  seed  RF-line  using  DP. 

-  Displacement  continuity  term  for  line  j: 

Rj{di ,  k ,  di-iji-i)  =  aa(cii  -  di- 1)2  +  ai(li  -  h-i)2 

-  Regularized  cost  function  at  the  zth  sample  of  the  /th  A-line: 

Ciiaidui)  =  [h(i,j)  - 10  +  <Hj  +  It)]2  +  min*.,*  +  wR.(ai, lu da, d,)} 

2.  Calculate  sub-sample  axial  and  lateral  displacements  for  the  seed  RF-line 
using  2D  Analytic  Minimization. 

3.  Propagate  the  solution  of  the  seed  RF-line  to  the  left  and  right,  using  the 
displacement  of  the  previous  line  as  initial  estimate. 

-  Calculate  a{  and  /,•  such  as  (ai  +  A at,  lt  +  A lt)  gives  the  axial  and  lateral 
displacements  at  the  sample  i.  The  regularized  cost  function  becomes: 

Ci(Aa1)...,Aara,  Ah,...,Alm)  =  (ij)  -  h{i  -f*  +  A  a,  j  +  l,+  Al.t)f  + 

+a  (a*  +  A oi  -  Oi_ i  -  Aai-i)2  +  ^  ( U  +  AZi  -  h~ i  -  Mfh  (h  +  A h  -  Uj-ifj 

weight  for  smoothness;  a,  and  /?,’  are  regularization  terms  which  ensure  continuity 
in  displacements  with  respect  to  the  top  (axial  a),  and  top  -  left/right  (lateral  Pa  and  /?,  ’ ) 


PROBLEM  -  Faulty  seed  RF-lines: _ 

The  displacement  of  the  previous  line  will  affect  the  displacement  of  the 
next  line: 

•  bad  initial  estimate 

•  the  last  term  — »•  enforces  lateral  smoothness 

For  experimental  evaluation  we  palpated  a  breast  elastography  phantom 
with  a  10mm  lesion  and  three  times  stiffer  than  the  background.  RF  data 
was  acquired  with  a  7.27MHz  linear  array  at  a  sampling  rate  of  40MHz. 

•  Strain  images  using  the  AM-2D  method,  with  each  RF-line  as  a  seed, 
each  for  1 1  (eleven)  values  for  w  (smoothness  regularization 


METHODS: 


Figure  2.  Examples  of  artifact  lesions  in  strain  images  using  DP  AM2D  method.  Data  collected  from  breast 
phantom  freehand  palpation. 


A  change-in-slope  parameter  was  computed  for  the  DP  displacement 
profile  for  each  seed  RF-line,  for  w=  0:25;  0:30;  0:35. 

486  lines  were  evaluated  for  each  of  the  three  values  for  w,  for  a  total  of 
1458  computations.  Positive  change-in-slope  test:  3  or  more  positions 
exhibited  a  continuous  change  in  the  slope  of  displacement  (Figure  3). 


Figure  3:  Change-in-slope  detection  algorithm.  The  3 
(three)  positions  where  the  change  in  slope  is  exhibited 
could  be  consecutive  (a)  or  not  (b) 


2.  DISPLACEMENT  STABILITY 


Table  2:  Change-in-slope  predictor  (TP  =  true  positive, 
FP  =  false  positive,  TN  =  true  negative,  FN  =  false 
negative) 

Sensitivity  =  91.7% 


Hypothesis:  A  robust  seed  RF-line  will  exhibit  the  same  or  similar  DP 
displacement  estimation  across  different  w  values. 

Displacement  Stability  test:  percentage  of  positions  (pixels)  where 
displacement  values  differ  by  more  than  3  pixels  for  different  values  of 
w.  For  all  486  lines,  3  (three)  pairs  of  displacement  values  were 
compared:  1)  w  =  0:25  vs.  w  =  0:30,  2)  w  =  0:30  vs.  w  =  0:35,  3)  w  = 
0:25  vs.  w  =  0:35.  Average  the  score  for  each  line;  final  score  between 
0  and  1 . 


Combined  score:  the  avg.  Change-in-Slope  score  over  the  3 
values  for  w  ( 0,  0.33,  0.66,  or  1)  was  added  to  the 


Displ.  Stability  +  Change-in-Slope 


MODIFIED  DP  +  AM2D  ALGORITHM 

Select  a  robust,  stable  seed  RF-line  using  random  search 

1 .  DP  integer  displacement  is  calculated  for  5  random  RF- 
lines,  each  at  5  random  w  values; 

2.  A  combined  Change -in -Slope  average  plus  Displacement 
Stability  average  score  is  computed; 

3.  The  most  robust,  stable  line  is  chosen  as  the  line  with  the 
smallest  combined  score,  which  also  does  not  have  any 
positive  Change-in-Slope  score; 

4.  The  chosen  sect  line  s  d  splacement  values  are  propagated 


with  AM2D. 


f  CONCLUSIONS: 

We  proposed  and  successfully  implemented  an  algorithm  for  the  selection  of  a 
robust,  stable  RF-line  to  be  used  as  seed  for  the  DP  displacement  estimation  and 
later  propagated  using  the  AM2D  algorithm  for  elastography.  The  benefit  of  this 
algorithm  is  significant  as  it  has  the  potential  to  improve  the  robustness  of 
ultrasound  elastography  in  in-vivo  tissue  which  can  be  highly  decorrelated.  The 
selection  of  robust  seed  RF-lines  becomes  even  more  important  as  we  move 
towards  real-time  3D  ultrasound  elastography. _ y 
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ABSTRACT 

Surgical  robots  provide  many  advantages  for  surgery,  including  minimal  invasiveness,  precise  motion,  high 
dexterity,  and  crisp  stereovision.  One  limitation  of  current  robotic  procedures,  compared  to  open  surgery,  is  the  loss 
of  haptic  information  for  such  purposes  as  palpation,  which  can  be  very  important  in  minimally  invasive  tumor 
resection.  Numerous  studies  have  reported  the  use  of  real-time  ultrasound  elastography,  in  conjunction  with 
conventional  B-mode  ultrasound,  to  differentiate  malignant  from  benign  lesions.  Several  groups  (including  our  own) 
have  reported  integration  of  ultrasound  with  the  daVinci  robot,  and  ultrasound  elastography  is  a  very  promising 
image  guidance  method  for  robot-assisted  procedures  that  will  further  enable  the  role  of  robots  in  interventions 
where  precise  knowledge  of  sub-surface  anatomical  features  is  crucial.  In  this  paper,  we  present  a  novel  robot- 
assisted  real-time  ultrasound  elastography  system  for  minimally  invasive  robot-assisted  interventions.  Our  system 
combines  a  daVinci  surgical  robot  with  an  experimental  software  interface,  a  robotically  articulated  laparoscopic 
ultrasound  probe,  and  our  GPU-based  elastography  system.  Elastography  and  B-mode  ultrasound  images  are 
displayed  as  picture-in-picture  overlays  in  the  daVinci  console.  Our  system  minimizes  dependence  on  human 
performance  factors  by  incorporating  computer-assisted  motion  control  that  automatically  generates  the  tissue 
palpation  required  for  elastography  imaging,  while  leaving  high-level  control  in  the  hands  of  the  user.  In  addition  to 
ensuring  consistent  strain  imaging,  the  elastography  assistance  mode  avoids  the  cognitive  burden  of  tedious  manual 
palpation.  Preliminary  tests  of  the  system  with  an  elasticity  phantom  demonstrate  the  ability  to  differentiate 
simulated  lesions  of  varied  stiffness  and  clearly  delineate  lesion  boundaries. 

DESCRIPTION  OF  PurPOSE  /  NEW  WORK  PRESENTED 

Surgical  robots  provide  many  advantages  for  surgery,  including  minimal  invasiveness,  precise  motion,  high 
dexterity,  and  crisp  stereovision.  One  limitation  of  current  robotic  procedures,  compared  to  open  surgery,  is  the  loss 
of  haptic  information  for  such  purposes  as  palpation,  which  can  be  very  important  in  minimally  invasive  tumor 
resection.  Numerous  studies  have  reported  the  use  of  real-time  ultrasound  elastography,  in  conjunction  with 
conventional  B-mode  ultrasound,  to  differentiate  malignant  from  benign  lesions  in  prostate,  breast,  pancreas,  lymph 
nodes,  and  thyroid. [1, 2,3 ,4,5]  Ultrasound  elastography  is  thus  a  very  promising  image  guidance  method  for  robot- 
assisted  procedures  that  will  further  enable  the  role  of  robots  in  interventions  where  precise  knowledge  of  hidden 
anatomical  features  is  crucial.  Several  groups  (including  our  own)  have  reported  integration  of  ultrasound  with  the 
daVinci  robot.[6,7,8]  In  this  paper,  we  present  a  novel  robot-assisted  real-time  ultrasound  elastography  system  for 
minimally  invasive  robot-assisted  interventions. 


METHODS 

In  general,  cancerous  tissue  has  higher  cell  density  than  surrounding  healthy  tissue,  which  leads  to  elevated  tumor 
stiffness  that  can  be  visualized  using  elastography  techniques.  As  first  described  by  Ophir  et  al.,[9]  the  principle  of 
ultrasound  elastography  is  to  estimate  tissue  stiffness  from  measurement  of  tissue  strain  induced  by  an  applied  force. 
This  is  accomplished  by  measuring  relative  tissue  displacements  between  ultrasound  image  pairs  under  different 
states  of  induced  compression.  Because  ultrasound  elastography  algorithms  assume  an  axial  compression,  strain 
image  quality  may  degrade  when  non-axial  motion  or  probe  rotation  occurs  during  compression.  Axial  compression 
is  typically  on  the  order  of  only  a  millimeter  or  two,  and  even  small  amounts  of  unwanted  motion  may  affect  image 
quality.  Due  to  the  variability  in  manual  compression,  image  quality  for  free-hand  elastography  is  highly  dependent 
on  practitioner  skill  and  experience. 

The  quality  of  elastography  imaging  as  a  function  of  human  performance  is  minimized  by  our  robot-assisted 
elastography  system  through  computer-integrated  motion  control  for  tissue  compression,  whereby  tissue  is 
autonomously  compressed  along  the  direction  axial  to  the  ultrasound  probe.  The  system  computer  provides  assistive 


control  of  robot  motion  by  generating  a  sinusoidal  palpation  motion  that  is  overlaid  onto  motion  commands  from  the 
master  manipulators  (Error!  Reference  source  not  found.).  By  this  method,  the  surgeon  retains  overall  control  of 
the  ultrasound  probe  position  and  orientation,  while  the  computer  controls  the  finer  points  of  tissue  compression. 
Because  the  computer  has  accurate  knowledge  of  the  ultrasound  probe  position  and  is  not  subject  to  motion  errors 
like  a  human  operator,  consistent  tissue  compression  in  the  precise  axial  direction  is  ensured. 


The  elastography  assistance  mode  improves  the  precision  and  consistency  of  tissue  compression  beyond  what  can  be 
achieved  by  manually  controlled  teleoperation  while  also  reducing  the  cognitive  burden  for  the  user.  Continual 
palpation  of  tissue  under  manual  control  is  a  tedious  task  imposing  a  large  cognitive  burden  and  demand  of  focus. 

By  relieving  this  burden  through  computer  assistance,  the  user  is  able  to  focus  on  more  important  tasks  such 
interpreting  real-time  imaging  information  and  conducting  surgery. 
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Figure  1.  Block  diagram  of  our  robot-assisted  system  for  real-time  ultrasound  elastography. 


Error!  Reference  source  not  found,  shows  a  block  diagram  of  our  complete  system  for  robot-assisted  ultrasound 
elastography.  At  the  heart  of  the  system  is  the  daVinci  S  surgical  robot  (Intuitive  Surgical  Inc.,  Sunnyvale,  CA). 
Computer  integrated  control  of  robot  motion  is  facilitated  by  an  experimental  “Read/Write”  Research  Application 
Programming  Interface  (API)  provided  by  Intuitive  Surgical  Inc.,  which  enables  robot  motion  to  be  controlled  from 
computer  in  addition  to  control  inputs  from  the  master  console,  as  discussed  below.  The  daVinci  robot  manipulates  a 
prototype  version  of  a  robotically  articulated  laparoscopic  ultrasound  probe  (Figure  2a),  also  developed  by  Intuitive 
Surgical  Inc.,  which  was  built  into  the  form  factor  of  a  standard  daVinci  tool.  The  ultrasound  probe  has  a  linear  array 
transducer  (Gore,  Newark  DE)  with  128  elements  that  measures  50mm  in  length  with  a  probe  diameter  of 
approximately  9mm.  The  ultrasound  probe  is  driven  by  a  Sonix  RP  ultrasound  system  (Ultrasonix  Medical  Corp., 
Richmond  BC  Canada),  which  provides  an  Ultrasound  Research  Interface  granting  access  to  pre-beam  formed  RF 
data  from  the  ultrasound  probe.  Access  to  this  RF  data  is  critical  to  reaching  optimal  performance  from  elastography 
algorithms.  The  RF  data  is  processed  by  a  high-performance  external  NVIDIA  Tesla  series  GPU,  which  connects 
locally  to  the  Sonix  RP  system.  The  GPU  provides  sufficient  parallel  computing  power  to  generate  strain  images 
from  RF  data  in  real  time  using  an  elastography  algorithm  based  on  normalized  cross-correlation. [10]  The 
elastography  images  from  the  GPU  are  streamed  from  the  Sonix  RP  system  to  a  network  port  on  the  system’ s  central 
workstation.  The  workstation  also  receives  a  parallel  stream  of  conventional  B-mode  ultrasound  images  from  the 
Sonix  RP  machine  directly.  The  workstation  sends  both  image  streams  to  picture-in-picture  overlays  shown  in  the 
stereo  display  of  the  daVinci  console.  The  picture  overlays  enable  the  surgeon  to  observe  the  ultrasound  and 
elastography  image  feeds  in  real-time  without  releasing  control  of  the  robot  arms  or  diverting  attention  away  from 
the  task  at  hand. 


In  collaboration  with  Intuitive  Surgical,  the  Engineering  Research  Center  for  Computer-Integrated  Surgical  Systems 
and  Technology  (CISST  ERC)  at  Johns  Hopkins  University  has  developed  an  open-source  software  framework  for 


medical  robotics  and  computer  assisted  surgical  systems  research,  which  we  call  Surgical  Assistant  Workstation 
(S AW). [1 1,12,13,141  This  open-source  framework  is  an  extension  of  the  CISST  software  libraries  developed  at  Johns 
Hopkins  to  enable  rapid  application  development  by  providing  capabilities  including  basic  mathematics  and 
numerical  routines,  thread  execution  management,  inter-process  communication,  construction  and  control  of  video 
pipelines,  and  many  other  things. [15, 16,17,181  Although  the  software  components  providing  its  basic  capabilities  are  all 
open-source,  SAW  is  designed  to  be  compatible  with  proprietary  modules  through  module  wrappers  using  well- 
defined  interface  protocols.  In  particular,  we  have  developed  SAW  wrappers  that  provide  the  ability  to  interface 
with  the  read-only  and  read-write  research  interfaces  of  the  daVinci  robot.  The  capabilities  of  the  Read-Only 
daVinci  Research  Interface  include  the  ability  to  query  the  state  of  the  master  and  slave  robot  arms  and  of  user 
console  events. [14]  The  Read/Write  daVinci  Research  Interface  extends  these  features  with  the  capability  to  use 
software  to  command  motions  of  the  master  and  slave  robot  arms  and  to  trigger  user  events.  One  such  feature  allows 
us  to  superimpose  an  externally  computed  motion  onto  motion  inputs  from  the  master  manipulators.  We  use  this 
capability  in  our  system  to  create  computer  generated  motion  overlays  for  tissue  palpation.  Another  function  of 
SAW  allows  us  to  use  a  master  arm  controller  effectively  as  a  3D  mouse.  We  call  this  masters-as-mice  mode.  We 
use  this  mode  in  our  system  to  build  an  interactive  environment  for  the  daVinci  operator  to  control  and  interact  with 
video  overlays. 

The  SAW  software  package  is  implemented  on  the  central  workstation  of  our  system,  which  centralizes  processing 
for  user  interaction  with  image  overlays,  acquisition  of  real-time  image  streams,  and  implementation  of  the  control 
loop  generating  assistive  motion  for  tissue  palpation.  A  command  terminal  on  the  central  workstation  allows  the  user 
to  set  the  amplitude  and  frequency  of  tissue  palpation.  These  settings  may  be  updated  in  real-time.  The  terminal  also 
allows  palpation  to  be  activated/deactivated  at  will.  The  user  interacts  with  the  image  overlays  from  the  daVinci 
console  using  the  SAW  masters-as-mice  mode,  which,  for  the  daVinci  system,  is  activated  by  engaging  the  clutch 
foot  pedal  followed  by  a  double-pinch  of  both  master  manipulators  in  unison.  The  masters-as-mice  mode  activates 
an  interactive  menu  environment  in  which  the  daVinci  master  controllers  manipulate  virtual  cursors  in  the  daVinci 
console’s  stereo  display  (Figure  2a).  Menu  selections  are  made  by  moving  a  cursor  to  a  menu  icon  and  pinching  the 
corresponding  master  controller.  The  elastography  and  B-mode  ultrasound  image  overlays  are  activated  by  selecting 
the  appropriate  menu  icon  with  a  virtual  cursor  (pinching  master  controller  selects).  The  image  overlays  are  resized 
and  repositioned  in  the  daVinci  display  by  dragging  them  with  a  cursor.  Releasing  the  clutch  pedal  returns  the 
master  manipulators  to  control  of  the  slave  arms  and  maintains  the  image  overlays  in  the  user’s  field  of  view. 

Results 

Preliminary  tests  of  the  system  have  been  conducted  with  a  CIRS  Model  049  Elasticity  QA  Phantom,  which  has 
simulated  lesions  of  different  calibrated  stiffness.  Figure  2b  presents  a  snap-shot  of  the  daVinci  display  taken  during 
phantom  testing  with  image  overlays  displaying  the  real-time  imaging  results.  The  elastography  image  in  this  figure 
shows  clear  difference  in  contrast  between  a  soft  and  hard  lesion,  with  the  hard  lesion  appearing  darker  in  the  image. 
This  distinction  cannot  be  made  from  the  B-mode  ultrasound  image  in  this  figure.  The  elastography  image  also 
establishes  clear  delineation  of  lesion  boundaries,  which  would  not  be  discernible  from  B-mode  ultrasound  in  the 
case  of  isoechoic  lesions.  These  images  were  recorded  with  tissue  palpation  set  to  1mm  amplitude  and  1Hz 
frequency. 


CONCLUSION 

We  have  successfully  implemented  a  robot-assisted  system  for  minimally  invasive,  real-time  ultrasound 
elastography.  Our  system  provides  an  improvement  over  manual  elastography  techniques  by  unifying  motion 
commands  from  a  user  with  the  precision  and  accuracy  of  computer-assisted  motion  control  to  ensure  consistent  and 
precise  tissue  strain.  Our  approach  effectively  reduces  the  cognitive  load  of  the  human  operator  while  maintaining  a 
user’s  control  of  the  procedure.  Preliminary  tests  using  an  elasticity  phantom  demonstrate  the  system’s  capability  to 
generate  strain  images  in  real-time  that  can  be  used  to  delineate  simulated  lesion  boundaries  and  differentiate  lesions 
of  varying  stiffness. 
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Figure  2.  a)  Interactive  menu  environment  for  displaying  image  overlays  in  the  daVinci  console  display,  showing  an 
active  menu  with  two  virtual  mice  corresponding  to  the  left  and  right  master  manipulators  and  two  picture-in-picture 
image  overlays  (picture  taken  of  a  patient  side  monitor),  b)  View  of  the  daVinci  console  display  during  a  test  with  an 
elasticity  phantom;  the  elastography  image  overlay  differentiates  lesions  of  different  stiffness. 

THIS  WORK  HAS  NOT  BEEN  SUBMITTED  FOR  PUBLICATION  OR  PRESENTATION  ELSEWHERE. 

REFERENCES 


[1]  Kamoi,  K.,  Okihara,  K.,  Ochiai,  A.,  Ukimura,  O.,  Mizutani,  Y.,  Kawauchi,  A.,  Miki,  T.,  “The  utility  of  transrectal  real-time  elastography  in 
the  diagnosis  of  prostate  cancer,”  Ultrasound  in  Medicine  &  Biology  34(7),  1025-1032  (2008). 

[2]  Tan,  S.,  Teh,  H.,  Kent  Mancer,  J.,  and  Poh,  W.,  “Improving  B  mode  ultrasound  evaluation  of  breast  lesions  with  real-time  ultrasound 
elastography-A  clinical  approach,”  The  Breast  17(3),  252-257  (2008). 

[3]  Hong,  Y.,  Liu,  X.,  Li,  Z.,  Zhang,  X.,  Chen,  M.,  and  Luo,  Z.,  “Real-time  ultrasound  elastography  in  the  differential  diagnosis  of  benign  and 
malignant  thyroid  nodules,”  J.  Ultrasound  Med.  28(7),  861-867  (2009). 

[4]  Saftoiu,  A.,  Vilmann,  P.,  Hassan,  H.,  Gorunescu,  F.,  “Analysis  of  endoscopic  ultrasound  elastography  used  for  characterisation  and 
differentiation  of  benign  and  malignant  lymph  nodes,”  Ultraschall  in  Med  27(6),  535-542  (2006). 

[5]  Saftoiu,  A.,  Vilmann,  P.,  Gorunescu,  F.,  Gheonea,  D.,  Gorunescu,  M.,  Ciurea,  T.,  Popescu,  G.,  Iordache,  A.,  Hassan,  H.,  and  Iordache,  S., 
“Neural  network  analysis  of  dynamic  sequences  of  EUS  elastography  used  for  the  differential  diagnosis  of  chronic  pancreatitis  and 
pancreatic  cancer,”  Gastrointestinal  Endoscopy  68(6),  1086-1094  (2008). 

[6]  Leven,  J.,  Burschka,  D.,  Kumar,  R.,  Zhang,  G.,  Blumenkranz,  S.,  Dai,  X.,  Awad,  M.,  Hager,  G.,  Marohn,  M.,  Choti,  M.,  Hasser,  C.,  and 
Taylor,  R.,  “DaVinci  canvas:  A  telerobotic  surgical  system  with  integrated,  robot-assisted,  laparoscopic  ultrasound  capability”,  MICCAI 
2005,  LNCS  Vol.  3749,  811-818  (2005). 

[7]  Schneider,  C.,  Dachs,  G.,  Hasser,  C.,  Choti,  M.,  DiMaio,  S.,  and  Taylor,  R.,  “Robot-Assisted  Laparoscopic  Ultrasound,”  IPCAI 2010, 
LNCS  Vol.  6135,  67-80  (2010). 

[8]  Schneider,  C.,  Guerrero,  J.,  Nguan,  C.,  Rohling,  R.,  and  Salcudean,  S.,  “Intra-operative  “pick-up”  ultrasound  for  robot  assisted  surgery  with 
vessel  extraction  and  registration:  A  feasibility  study”,  IPCAI  2011,  LNCS  Vol.  6689,  122-132  (2011). 

[9]  Ophir,  J.,  Cespedes,  I.,  Ponnekanti,  H.,  Yazdi,  Y.,  and  Li,  X.,  “Elastography:  A  quantitative  method  for  imaging  the  elasticity  of  biological 
tissues”,  Ultrasonic  Imaging  13(2),  111-134  (1991). 

[10]  Deshmukh,  N.,  Rivaz,  H.,  and  Boctor,  E.,  “GPU-based  elasticity  imaging  algorithms”,  Proc.  Int.  Conf.  Med.  Imag.  Comp.  &  Comp.  Assist. 
Interven.,  (2009). 

[11]  Jung,  M.,  Deguet,  A.,  and  Kazanzides,  P.,  "A  component-based  architecture  for  flexible  integration  of  robotic  systems",  IROS  2010,  6107- 
6112(2010). 

[12]  Jung,  M.,  Xia,  T.,  Deguet,  A.,  Kumar,  R.,  Taylor,  R.,  and  Kazanzides,  P.,  “A  Surgical  Assistant  Workstation  (SAW)  application  for 
teleoperated  surgical  robot  system”,  Sys.  &  Arch,  for  Comp.  Assist.  Interven.,  Midas  Journal,  (2009). 

[13]  Balazs,  V.,  Simon,  D.,  Anton,  D.,  Peter,  K.,  Rajesh,  K.,  Christopher,  H.,  and  Russell,  T.,  “The  Surgical  Assistant  Workstation”,  MICCAI 
2008  Workshop,  Midas  Journal,  (2008). 

[14]  DiMaio,  S.  and  Hasser,  C.,  “The  da  Vinci  research  interface”,  MICCAI  2008  Workshop,  Midas  Journal,  (2008). 

[15]  Kazanzides,  P.,  Deguet,  A.,  Kapoor,  A.,  "An  architecture  for  safe  and  efficient  multi-threaded  robot  software",  TePRA  2008,  89-93  (2008). 

[16]  Deguet,  A.,  Kumar,  R.,  Taylor,  R.,  and  Kazanzides,  P.,  “The  cisst  libraries  for  computer  assisted  intervention  systems”,  MICCAI  2008 
Workshop,  Midas  Journal,  (2008). 

[17]  Kapoor,  A.,  Deguet,  A.,  Kazanzides,  P.,  "Software  components  and  frameworks  for  medical  robot  control,"  ICRA  2006,  3813-3818  (2006). 


[18]  Kazanzides,  P.,  Deguet,  A.,  Kapoor,  A.,  Sadowsky,  O.,  LaMora,  A.,  and  Taylor,  R.,  “Development  of  open  source  software  for  computer- 
assisted  intervention  systems”,  2005  MICCAI  Open-Source  Workshop,  Insight  Journal,  (2005). 


